Almost every AI rescue engagement we take on has the same shape. A pilot was built, it demonstrated well, a business case was written on the back of it, and then nothing happened for a year. Nobody killed it. It simply never became a system anybody depends on.
The failure is rarely the model. It is almost always one of four things.
1. There was no decision attached
The pilot answered a question nobody was going to act on. "Can we predict churn?" is not a business decision. "Should this account go to the retention team this week, and what does it cost us if we are wrong?" is. Without a decision, there is no owner, no threshold, and no way to say whether the model is good enough — so it never is.
Before we build anything we write down the decision, who makes it today, what they use now, and the number that will prove the model improved it. If we cannot fill in all four, the project does not start.
2. The baseline was never measured
A model with 87% accuracy sounds good until you discover the person currently doing the job is at 94%. This happens more often than anyone admits, because the human baseline is rarely measured — it is assumed to be bad, which is why the project was funded.
Measure it first. Sometimes the honest answer is that the humans are excellent and the opportunity is throughput rather than accuracy. That is still a valuable project; it is just a different one, with a different design.
3. Production was treated as a deployment step
The pilot ran in a notebook against a clean extract. Production means authentication, rate limiting, caching, cost ceilings, retry semantics, observability, a rollback path, and an answer to what happens at three in the morning when the upstream system returns malformed data.
That work is typically larger than the modelling. Teams that budget for a two-week "deployment phase" after a successful pilot discover this at exactly the wrong moment. We budget production at roughly the same effort as the model itself, and we have never regretted it.
4. Nobody owned it after handover
Models decay. The data distribution shifts, an upstream system changes format, a business rule changes and nobody tells the model. Without a named owner, a monitoring dashboard people actually look at, and a retraining trigger, the system degrades quietly until somebody notices it is wrong and switches it off.
The single highest-leverage thing we build is not the model. It is the close-out loop: every prediction the system makes gets marked by a human as correct, incorrect or inconclusive, and that feedback goes straight back into evaluation. Systems with that loop keep improving after we leave. Systems without it start dying the day we do.
What this looks like in practice
On a recent predictive-maintenance engagement we spent the first six weeks doing nothing but reconciling eighteen months of failure history against sensor data — establishing which failures had actually been detectable in advance. Roughly a third had not been. That number went into the business case, which meant the target was achievable rather than aspirational, and the project was judged a success rather than a disappointment.
That reconciliation was unglamorous, produced no demo, and was the reason the system is still running.
The uncomfortable summary
Most stalled AI projects were well-executed answers to a question nobody needed answered, evaluated against a baseline nobody measured, deployed into an environment nobody prepared, and owned by nobody afterwards.
None of those are modelling problems.