In short: Most AI pilots stall silently. The five observable signals are: (1) no evaluation harness beyond the demo, (2) the dataset is curated rather than production-representative, (3) no named internal owner with delivery authority, (4) "it works in the demo" is the highest bar it has ever met, and (5) the pilot has sat in "promising" status for a quarter or longer. If three or more are true for your project, the pilot is statistically unlikely to reach production without an intervention. S&P Global Market Intelligence's Voice of the Enterprise: AI & Machine Learning, Use Cases 2025 (survey fielded Oct–Nov 2024, published October 2025, n=1,006 IT and line-of-business professionals across North America and Europe) found that 42% of organisations abandoned most of their AI initiatives in 2025, up from 17% the year before. Stalling is the norm. Diagnosing it early is the differentiator.
What does a stalled AI pilot look like?
A stalled AI pilot is one that has demonstrated value in a controlled setting but has not moved measurably closer to production in 60 or more days.
The distinction between "stalled" and "failed" matters operationally. A failed pilot has been assessed against production criteria and found unable to meet them — a decision was made. A stalled pilot has never been assessed against production criteria at all. It persists in a pre-gate state: showing well, generating stakeholder confidence, clearing no new bars. The team keeps proving itself to the same internal audience. The evaluation is the demo. The demo is the output. Nothing accumulates toward production.
This is more common than most teams admit. The S&P Global data cited above is instructive not just for its scale but for its direction: the abandonment rate nearly tripled in a single year, from 17% to 42%. The companion figure from the same study — that organisations scrapped an average of 46% of proof-of-concept projects before broad adoption — places the discontinued pilot in the majority rather than the margin.
The tone to adopt here is diagnostic, not catastrophic. If your pilot is stalled, you are in the majority. The question is whether you name it early enough to act.
(For the root causes that drive pilots into this position in the first place, see why AI pilots fail to reach production — that piece covers the upstream design and scoping errors; this one is the mirror you hold up to a project already underway.)
How long is too long for a pilot to stay "promising"?
In practice, a pilot that remains in "promising" status beyond 90 days without clearing a single production-readiness criterion has entered stall territory.
The 90-day mark is a diagnostic threshold, not a rule. Calendar time is a proxy for the more meaningful question: has any production criterion — security review, latency benchmark, data governance sign-off, integration test against live systems — been cleared in the last 30 days? If the answer is no, the pilot is not progressing. It is being maintained.
No verified primary source gives a clean "average pilot dwell time" that meets the evidence standards used throughout this piece, so the time signal is kept qualitative. What the data does establish clearly is the conversion gap. IDC research conducted with Lenovo (CIO Playbook 2025: It's Time for AI-nomics, February 2025, surveying 900 IT and business decision-makers across Asia-Pacific mid-to-large organisations) found that for every 33 AI proofs-of-concept a company launched, roughly four reached production. The accumulated remainder is the set of pilots this piece is about: alive on the books, going nowhere in fact. Deloitte's State of AI in the Enterprise 2026 (survey fielded August–September 2025, n=3,235 business and IT leaders across 24 countries) found that only 25% of organisations have moved 40% or more of their AI experiments into production.
Gartner forecast in July 2024 that at least 30% of generative AI projects would be abandoned after proof of concept by end of 2025, citing poor data quality, inadequate risk controls, escalating costs, and unclear business value — supporting the conversion gap without serving as the primary evidence here.
The organisational dynamic is worth naming. The longer a pilot stays in "promising," the harder it becomes to cancel or restructure — sunk-time reasoning calcifies around it. Teams describe it in retrospect as having been "almost ready for months." The stall does not announce itself. It accumulates.
The 5 signals your AI pilot won't reach production
These five signals are observable before a pilot runs out of budget or goodwill — and three or more in combination is a reliable indicator that production is not the current trajectory.
A note on structure: these signals are not independent. They form a causal chain. No exit criterion (Signal 5) is definable because there is no evaluation harness (Signal 1). No harness means quality is judged by the demo (Signal 4), which runs on curated inputs (Signal 2), because no one owns the question of whether it holds up on real traffic (Signal 3). The root is a single missing artifact: a measurable definition of "working" that runs on representative data and is owned by someone. Each signal is that absence viewed from a different angle.
Signal 1 — No evaluation harness beyond the demo
Definition: The only measurement of the system's performance is whether stakeholders liked a presentation.
Underlying cause: Success criteria were defined for the demonstration, not for the production environment. Quality lives in someone's head, not in a scoring function.
Why it matters technically: Without an evaluation harness — an offline test set with known-correct answers, a scoring function, and a regression gate — you cannot detect deterioration. The moment you change a prompt, a model version, a retrieval source, or a preprocessing step, you have no way to know whether you made the system worse. Production systems are not static; model deprecations alone force changes. A system that cannot measure whether a change helped is one that can only drift. Google's ML Test Score rubric (Breck, Cai, Nielsen, Salib, and Sculley, The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction, IEEE Big Data, 2017) defines production readiness through the presence of 28 specific tests and monitoring checks across data, model, infrastructure, and monitoring domains. "It looks good" scores zero on that rubric.
The eval harness is not a QA nicety added at the end of development. It is the instrument that makes iteration possible. Teams without one do not iterate slowly — they cannot iterate at all. Which is precisely why they sit in "promising" for quarters.
Observable check: Ask the team: "What is your current score on your evaluation set, and what was it a month ago?" If there is no number, or the number is not written down anywhere, you have your answer — independent of how impressive the demo is.
Signal 2 — The dataset is curated, not production-representative
Definition: The data the pilot runs on was selected or cleaned for the purpose of the demonstration; the same examples appear in every run.
Underlying cause: The system was tuned, implicitly or explicitly, on the very inputs it is shown succeeding on. This is leakage. The evaluation no longer measures generalisation — it measures the curator's taste.
Why it matters technically: Real production traffic is a different distribution from any curated sample — longer inputs, messier formatting, adversarial edge cases, the entries the curator quietly removed. A model can perform excellently on a chosen set and fail systematically on live traffic. Paleyes, Urma, and Lawrence's survey of machine learning deployment case studies (Challenges in Deploying Machine Learning: A Survey of Case Studies, ACM Computing Surveys, 2022) identifies distribution mismatch between development data and production data as one of the recurring, named failure modes across real deployments.
The practical implication: performance on a hand-picked input set is not weak evidence of production-readiness. On a curated set it is no evidence at all, because curation is exactly what hides the failure modes. Curated golden sets are correct and necessary as one evaluation layer — the signal here is when the curated set is the only evaluation and doubles as the demo input.
Observable check: Ask to run the system, live, on an input the questioner supplies on the spot — something not previously shown to the team. Watch whether the team flinches before running it. The flinch is the diagnosis.
Signal 3 — No named internal owner with delivery authority
Definition: There is enthusiasm and sponsorship, but no individual with budget authority, a mandate to ship, and accountability for a go-live date.
Underlying cause: The pilot was built by technical champions who lack the organisational authority to clear cross-functional blockers — legal sign-off, security review, procurement, integration with downstream systems. Sponsorship and ownership are different things, and the pilot has one but not the other.
Why it matters technically: Machine learning systems decay without intervention even when no code changes, because the world around them changes. The research literature on deployed systems formalises this as concept drift — when the statistical relationship between inputs and the target shifts over time, deployed models silently lose accuracy as their environment diverges from training conditions (Gama, Žliobaitė, Bifet, Pechenizkiy, and Bouchachia, A Survey on Concept Drift Adaptation, ACM Computing Surveys, 2014). A system with no owner has no one to notice the decay, no one to trigger retraining, no one to be alerted when it breaks. It is structurally abandoned on arrival.
Sculley et al. (Hidden Technical Debt in Machine Learning Systems, NeurIPS, 2015) found that only a small fraction of a real-world ML system is composed of the ML code itself — the surrounding infrastructure of data collection, verification, configuration, monitoring, and serving dwarfs it, and systems acquire undeclared consumers and unmonitored dependencies precisely because no one owns the boundaries. Ownership is not an org-chart concern layered on top of technical work. It is a reliability condition built into the technical work.
Observable check: Ask: "Who gets paged when the system returns a wrong answer in production, and how would they know it had?" If the answer is silence, or "we would hear about it from a user," there is no owner.
Signal 4 — "It works in the demo" is the highest bar the pilot has ever cleared
Definition: Every conversation about the pilot is a re-run of the same demonstration. No harder question has been asked, and no means to answer one has been built.
Underlying cause: Signals 1 and 2 have compounded into a culture. The team has never been asked — and never built the instrumentation — to answer any harder question than "can it produce one good output while watched." The demo bar and the production bar are separated by everything that does not appear in a demo: integration with live systems, latency and cost under load, error handling for edge cases, failure-mode behaviour, monitoring, rollback capability, and verification that outputs are correct rather than merely plausible.
Plausible-but-wrong outputs are the single most dangerous failure class in generative systems. A demo specifically selects against ever surfacing one.
What practitioners recognise but brochures miss: the polish of a demo is negatively correlated with production-readiness in stalled pilots, because polish is where the remaining discretionary effort went. A compelling demo with no harness, no live-data test, and no owner is not "almost there." It is at the beginning of the production-readiness work, having spent its credibility appearing to be at the end.
Observable check: List the production requirements the system must clear before deployment — latency under SLA, security review, integration tests, error-handling coverage, rollback plan. Score the pilot against each. The number of cleared items is the honest progress measure. If the list has never been written, that is Signal 4 confirmed.
Signal 5 — The pilot has been "promising" for a quarter or longer
Definition: More than 90 days have passed without a single production-readiness criterion being cleared.
Underlying cause: "Promising" with no improving metric is the natural resting state of a pilot with no evaluation harness. There is nothing to push it forward, and nothing to declare it done, so it stays. The IDC / Lenovo 2025 research cited above quantifies the structural version of this: for every 33 AI proofs-of-concept launched, roughly four reach production. The remainder do not fail on a specific criterion — they simply never define one.
Time-in-pilot is not a neutral cost. It is evidence of the absence of an exit criterion. A healthy pilot has a written, measurable "done" — a target score on the evaluation set, a latency and cost budget, a go/no-go gate. The reason a pilot can remain "promising" for a quarter is that no such number was ever defined. Duration is the symptom; the missing exit criterion is the disease.
Observable check: What was the last hard decision made about this pilot, and when? "We decided to continue" is not a hard decision. "We set a 30-day window to clear the security review or restructure scope" is. If no decision date surfaces, the pilot has no clock running.
Signal → cause → next step: a quick-reference table
Each signal points to a specific underlying cause; knowing the cause determines whether the pilot is recoverable and what intervention it needs.
| Signal | Root cause | Recoverable? | Fastest first step |
|---|---|---|---|
| No evaluation harness | Success criteria defined for demo, not production | Yes, if addressed within current sprint | Define three automated, production-representative test cases this week |
| Curated dataset only | Production data complexity was not scoped in | Yes, with a data audit | Pull 200 unfiltered records from the live system and run the model against them |
| No named owner | Sponsorship was treated as equivalent to ownership | Yes, with an explicit mandate | Assign a named owner with a go-live date in writing before the next sprint |
| "Works in demo" only | Production requirements were never formalised | Yes, with a requirements gate | List all production criteria; score the pilot against each one publicly |
| 90+ days "promising" | No escalation trigger or exit criterion exists | Conditionally | Set a 30-day decision checkpoint: proceed with the current scope, restructure, or close |
Recoverability is conditional on the fundamentals being sound. A pilot stalled on Signal 3 — no named owner — is often unblocked in days with a single organisational decision. A pilot stalled on Signal 2 with severely unrepresentative training data may require a scope redesign that is functionally a restart. The triage purpose is to distinguish between those two cases before additional cycles are spent.
My pilot is stalled — what do I do next?
The first move is a structured triage: not a new vendor evaluation, not a rebuild, but a focused assessment of whether the pilot's fundamentals are sound enough to be worth recovering.
A triage has three possible outcomes. First, the pilot is recoverable with the current team and scope, with a clear remediation plan and a production timeline that can now be committed to. Second, the pilot is recoverable but requires an external senior review to unblock a specific gap — an evaluation harness that has not been built, a data audit that requires access the internal team does not have, or an ownership decision that requires authority above the project level. Third, the pilot is not recoverable in its current form — the original business problem has materially changed, or the scoping assumptions were wrong at the foundation — and faster progress comes from a redesigned brief than from repairing what exists.
What a triage is not: a sales conversation dressed as a diagnosis. Its function is to give the leadership team a clear verdict, stated plainly, before they spend more budget and more goodwill on a project that may not be salvageable. The verdict should be the same whether or not the diagnosing party has a commercial interest in the outcome. That independence is the point: the same discipline that separates a working production system from a convincing demo — measure it, run it on real inputs, verify the result rather than assert it — is what separates an honest triage from a pitch.
The counter-evidence is worth stating here: stalling is not permanent, and the majority of organisations expect to improve. Deloitte's 2026 survey found that while only 25% of organisations had moved 40% or more of their AI experiments to production at the time of surveying, 54% expected to reach that level within three to six months. BCG's The Widening AI Value Gap (September 2025) found that AI leaders — the top 20% of organisations — achieve five times the revenue increases and three times the cost reductions of laggards. The structural gap between stalled and producing is real, but it is not a one-way door. The organisations that cross it are the ones that stop managing the stall and start measuring against a definition of "in production, and proven to work."
If three or more of the five signals are true for your project, a triage will tell you whether it is recoverable. AI Rescue starts with exactly that diagnosis. Or book a 30-minute session to discuss where your pilot currently stands.
If you are also weighing whether to bring in an external team to do that assessment, how to choose an AI partner covers what to look for in a credible triage process and how to vet the firm conducting it.
Common questions
What percentage of AI pilots fail to reach production?
S&P Global Market Intelligence's Voice of the Enterprise: AI & Machine Learning, Use Cases 2025 (published October 2025, n=1,006 IT and line-of-business professionals across North America and Europe) reported that 42% of organisations abandoned most of their AI initiatives in 2025, up from 17% the year prior. The same study found organisations scrapped an average of 46% of proof-of-concept projects before broad adoption. While "stalled" and "abandoned" are not identical, the data suggests that the majority of pilots that do not reach production are discontinued rather than formally assessed and recovered. IDC's 2025 enterprise research with Lenovo (CIO Playbook 2025, February 2025) puts the production conversion rate in structural terms: for every 33 AI proofs-of-concept launched, roughly four reach production.
Is a stalled AI pilot always worth trying to recover?
Not always. The triage question is whether the fundamental scoping and data assumptions are sound enough to build on. A pilot stalled on Signal 3 — no named owner — is often recoverable in days with a single organisational decision that was never made. A pilot stalled on Signal 2 — a dataset that does not represent production — may require a scope redesign that is functionally a restart. The signals are not uniformly fixable; the honest answer after a triage is sometimes "redesign with a clearer brief" rather than "continue with remediation."
How is a stalled AI pilot different from one that simply failed?
A failed pilot has been assessed against production criteria and found unable to meet them — a decision was made, even if the outcome was disappointing. A stalled pilot has never been assessed against production criteria at all. It persists in a pre-gate state indefinitely, still described as "promising," generating cycles without generating evidence. Stalling is more common than failure in the technical sense and more recoverable — but it requires an active intervention to break the loop, because nothing in the current structure will do so automatically.
Should I bring in an external team to diagnose a stalled pilot?
External review is most useful when the block is structural rather than purely technical — no named owner, no production requirements ever formalised, no clear budget path to deployment. An internal team close to the work often lacks the authority or the detachment to make the "stop, restructure, or continue" call. The people who built the demo have an understandable interest in it continuing. An independent triage preserves the authority to make that call cleanly, and produces a verdict the leadership team can act on without relitigating it.
At what point should I cancel an AI pilot rather than triage it?
If all five signals are present and the original business problem has materially changed since the pilot was scoped, cancellation with a redesigned brief is usually faster than attempting recovery. The 90-day "promising" signal (Signal 5) is the most common trigger for this conversation — not because duration is the deciding factor, but because duration is the visible symptom of the missing exit criterion, and a pilot that has never defined "done" has no natural point at which the question of cancellation becomes unavoidable. Setting one is the intervention.
NewGenApps — Stay a step ahead, always.