A pilot that produced an impressive demo and no answer on the six questions below has told you almost nothing about whether the thing is viable. That is an uncomfortable position, because the demo is the part everyone remembers and the part the business case gets built on.
The demo tests whether the model can do the task at all. That was rarely the thing in doubt. What decides whether a system survives contact with an operation is a different list, and it is a boring one.
1. Cost per transaction at volume
Pilot economics are calculated on pilot volumes, where the numbers are small enough that nobody looks closely. The question is what a single transaction costs when the traffic is a hundred times larger, and whether the answer still clears the saving it is supposed to produce.
This inverts more often than people expect, because the cheapest way to make a model accurate is usually to give it more context, more retries, or a second model to check the first. Each of those is a real answer to an accuracy problem and each one multiplies the unit cost.
2. Latency under real load
A response time that is perfectly acceptable while one person clicks through a demo can be unusable inside an operational workflow, where the system sits between a person and the next thing they have to do. The relevant measure is not the average. It is what happens at the busy hour, when the queue is longest and the people using it are least able to wait.
Latency also compounds with the containment work below. Every check that stops a model guessing — a second pass, a citation requirement, a rules engine standing between the model and the answer — costs time as well as money. The accurate version of a system is rarely the fast one, and the trade is easier to make deliberately than to discover in production.
3. Accuracy on messy real input
Curated test data flatters everything. Real inbound data is not curated: it arrives in the email body rather than the attachment, in formats that differ by sender, with the exceptions that never made it into the procedure.
Worse, the headline accuracy figure can be the thing hiding the error. In our own worked example, a document pipeline scored 82% field-level accuracy on its first pass while every monetary value it produced was wrong by two orders of magnitude. The score was not lying, exactly. It was measuring something other than whether the output was usable.
4. Hallucination containment
Not "does it hallucinate" — it does. The question is where in the process the model must never guess, and what structurally prevents it from guessing there.
That is an architecture decision, not a prompt. The usual shape is a hard boundary: the model extracts what a document states, and deterministic rules decide what that means. It keeps the consequential decision in code that can be read, tested and defended by someone who knows the domain rather than someone who knows prompts.
5. Evaluation and regression
How do you know a change improved things rather than breaking something else that used to work? Without an answer, every subsequent change is a gamble, and the system quietly degrades as it is improved.
The useful checks are often cheap. Requiring every extracted value to carry the exact line of text it was read from needs no labelled data and no domain expertise, and it catches the class of error where the model is confidently reporting something the document does not say.
6. Integration into the system of record
The unglamorous work that decides whether any of the rest counts. If the output lands in a spreadsheet that somebody retypes into the ledger, the saving has been removed and a step has been added.
This is where a surprising number of otherwise sound pilots stop. Nothing downstream consumes the output, so the benefit is real in the demo and unrealisable in the operation.
None of this is research
These are engineering and architecture problems. They do not need a breakthrough and they do not need a quarter — they need somebody to go looking for them deliberately, before the budget is committed rather than after.
If you can only test some of them, start with the ones that are cheapest to answer and most likely to be fatal. Unit cost at volume and integration into the system of record usually qualify on both counts: neither needs a finished model to investigate, and either one can end a business case on its own. There is no sense proving accuracy on a process whose output nothing downstream can accept.
That is the whole argument for finding out first. The six questions are answerable in weeks. The seven-figure commitment they inform is not reversible in weeks.