Nishaglobal Education logo

The AI project lifecycle

How AI projects move from idea to discovery, proof of concept, pilot and production, what makes them different from ordinary software projects, the go/no-go gates between stages, and the common reasons AI projects stall in "pilot purgatory".

Lesson 28 of 4417 min readQuiz: 5 questions

๐ŸŽฏ By the end of this lesson you will be able to:

  • Explain how AI projects differ from traditional software projects
  • Run an AI project through clear stages with evidence-based go/no-go gates
  • Spot and prevent the common reasons AI projects stall before production
In this lesson
  1. Why AI projects are different
  2. The five stages
  3. Go/no-go gates
  4. Why AI projects stall
  5. The people you need
  6. What this means for your role
  7. Key takeaways
  8. Quiz

Why AI projects are different

In a traditional project, if the team builds what was specified, it works. In an AI project, you can build everything correctly and still find the AI isn't good enough for the job. Quality is uncertain until you measure it with real data. That single fact changes how you plan, report and make decisions.

Traditional softwareAI features
Behaviour is specified and predictableBehaviour is probabilistic; quality is measured, not assumed
Done means the requirements are metDone means quality thresholds are met on an evaluation set
Risk is mostly scope, time and budgetPlus quality risk, data risk, model change and misuse
After launch, mainly bug fixesAfter launch, ongoing monitoring, evaluation and tuning
Cost mostly fixed (build, hosting)Running cost grows with usage (tokens, calls)

The five stages

Discoveryworth solving?Proof of conceptcan AI do it?Pilotreal users?Productionrun reliablyOperatekeep improvinginvestment grows only after each gate is passedgo/no-go gateeach gate: go ยท pivot ยท stop, decided on agreed evidence
Each stage ends with a gate. The evidence needed grows, and so does the investment, but only after the previous gate is passed.
StageGoalTypical lengthExit evidence
1. DiscoveryIs this a real problem worth solving with AI?1โ€“3 weeksProblem statement, users, baseline numbers, data available, success metric
2. Proof of conceptCan AI do the core task well enough?2โ€“6 weeksResults on a small evaluation set vs a target
3. PilotDoes it work for real users in real conditions?1โ€“3 monthsQuality, adoption, feedback and cost from a limited group
4. ProductionRun it reliably for everyoneOngoingMonitoring, support, runbooks, approvals in place
5. Operate and improveKeep value high as things changeOngoingRegular quality, cost and value reviews

Go/no-go gates

Agree the gate criteria before each stage starts, with numbers. Otherwise every result looks "promising" and projects drift on without a decision. A gate can say go, stop, or change direction (pivot), and stopping a project early is a success when the evidence says it won't work.

Gate criteria for a proof of concept

Use case: draft replies to customer emails about deliveries
Go if: โ‰ฅ 80% of drafts rated "usable with light edits" on 150 real past emails, zero drafts with wrong refund amounts, cost per draft < $0.02
Pivot if: 60โ€“80% usable โ†’ narrow scope to the 3 most common email types
Stop if: < 60% usable, or the order data needed isn't accessible

Everyone knows in advance what success looks like and what happens next.

Why AI projects stall

Many AI projects produce an impressive demo and then never reach production. This is sometimes called pilot purgatory. The usual causes are predictable, so you can plan against them.

CausePrevention
No clear business owner or success metricName an owner and a measurable outcome in discovery
Demo built on a handful of hand-picked examplesMeasure on a realistic evaluation set from the start
Data access, quality or permissions blocked lateCheck data access and approvals in discovery
Security, legal or risk review starts at the endInvolve them at the first gate, scaled to risk
No plan for integration, support and monitoringPlan production work from the pilot stage
Users don't change how they workInvolve users early; plan training and workflow change
Running cost unknown until scaleEstimate cost per use in the proof of concept

The people you need

  • Business owner: owns the problem, the success metric and the go/no-go decisions.
  • Domain experts: define good answers, label evaluation data and review outputs.
  • AI engineers and developers: build, integrate and tune.
  • Evaluation engineer or QA: builds the evaluation set and measures quality.
  • Architect: platform, security and integration choices.
  • Risk, security, legal and privacy: approvals proportional to risk.
  • Change and training: help users adopt the new way of working.

What this means for your role

Developer

Build the proof of concept to answer the gate question cheaply. Don't over-engineer it, but measure it honestly on realistic data.

Tester / QA

Create the evaluation set during discovery or early in the proof of concept; gates depend on it.

Business Analyst / PO

Own the problem statement, baseline numbers and the success metric in discovery, and keep them visible at every gate.

Tech Lead / Architect

Flag architecture, data access and security blockers in discovery, and design the pilot so it can grow into production.

Project / Delivery Manager

Run the project in stages with agreed gate criteria, report evidence rather than optimism, and treat an early stop as a valid outcome.

Scrum Master / Agile

Help the team work in short experiment cycles within each stage, and make the gate criteria part of the team's shared definition of done.

Try it: write gate criteria

  1. Pick an AI idea in your organisation that is at idea or demo stage.
  2. Write go, pivot and stop criteria for its proof of concept, with numbers for quality, safety and cost.
  3. List the three biggest reasons it could stall later, and one action to prevent each now.

Key takeaways

  • AI quality is uncertain until measured, so AI projects run in stages with evidence-based gates.
  • The five stages: discovery, proof of concept, pilot, production, operate and improve.
  • Agree go, pivot and stop criteria with numbers before each stage starts; stopping early is a valid outcome.
  • Prevent pilot purgatory with an owner, a metric, realistic data, early approvals, production planning and user adoption.

Key terms

Proof of concept (PoC)
A small, quick test of whether the core idea can work at all.
Pilot
A limited real-world rollout to a small group of users to test value and practicality.
Go/no-go gate
A decision point where evidence decides whether to continue, change direction or stop.
Pilot purgatory
When AI projects get stuck after a promising pilot and never reach production.

Check your understanding

Answer all 5 questions. Score 3 or more to complete the lesson.

  1. 1. What is the biggest difference between AI projects and traditional software projects?

  2. 2. When should go/no-go criteria for a stage be agreed?

  3. 3. A proof of concept looked great on 10 hand-picked examples. What is the risk?

  4. 4. When should security, legal and risk teams be involved?

  5. 5. The proof of concept clearly misses its targets. What is a good outcome?

Answer all 5 questions to check.

Practise more