The pattern is consistent enough to be predictable. Something gets built. It works, roughly. Some people like it and some people don't, and both positions are defensible from the same evidence. So the decision gets deferred pending more information that nobody is tasked with gathering, the project loses its sponsor's attention, and it is quietly forgotten.
Nobody records this as a failure, which is precisely the problem. A failure is informative. A pilot that was never judged tells you nothing, costs what it cost, and leaves the same question open for the next team to spend money on.
The cause is upstream of the technology
This happens because nobody agreed in advance what success would look like. Not vaguely — specifically, in writing, with the person who would have to act on the answer.
Without that, the evaluation becomes an argument about which numbers to look at, held after the numbers exist and after everyone involved has developed a position they would prefer to be right. That argument has no natural end, which is why it ends in deferral.
The fix is one page, agreed before anything is built
Five lines, signed off by the sponsor before the first line of code:
1. The metric that matters
Chosen with the operators who own the process, not with the technologist. The people doing the work know which number reflects whether their day got better. It is frequently not the number the project was named after.
2. Its baseline today
Measured from your systems, not estimated in a workshop. This step alone kills a meaningful number of proposals, because the current state turns out to be unmeasurable or considerably better than assumed.
3. The result that would justify the investment
Set by finance, not by the vendor. The threshold that makes this worth doing is a commercial judgement, and it should be made by the function that will have to defend it.
4. How it will be measured
Agreed method, agreed sample, agreed period. Written down, because the temptation to change any of the three after seeing the result is very strong and entirely human.
5. The threshold for scale, fix or stop
Three named outcomes with a number attached to each, written before anyone is emotionally invested in which one applies. This is the line that converts a result into a decision instead of a discussion.
Who signs it matters as much as what it says
One sponsor, not a committee. The page has to be agreed by the person who can actually stop the work, because its purpose is to make stopping possible without anyone having to lose an argument to do it. A page endorsed by five people who each read a different emphasis into it is simply the deferral arriving early.
The most common distortion is letting the technologist choose the metric. It is an understandable delegation — they are the ones who understand what the system can move — but it tends to produce a number the system can reliably improve rather than the number the business needed improved. Those are only sometimes the same number, and the difference is invisible until someone asks what changed commercially.
The objection, and the answer to it
The usual objection is that you cannot know the metric until you have built something and seen what it can do. Occasionally that is genuinely true.
When it is, the honest response is not to skip the page. It is to call the work research, budget it as research, and set a much smaller number against it. Research is a perfectly reasonable thing to fund. It is only dangerous when it is presented as a pilot and then expected to produce a decision it was never designed to support.
Why it is worth a week
It takes about a week to agree, and it is the highest-leverage document in the whole exercise, because it is what converts a prototype into an investment paper. Everything downstream — the build, the evaluation, the board conversation — inherits its clarity or its absence.
And if you cannot get that page agreed, that is itself the finding, and it is worth the week on its own. It means the value was never really specified, only assumed. No amount of engineering was going to rescue that, and the honest thing is to learn it before the build rather than after.