When Human Review Costs Less Than a Wrong AI Decision

An AI system does not have to decide every case it sees. For ticket routing, document classification or another repeatable operational decision, it can send uncertain cases to a person. The useful question is which cases should receive that review, given the cost of a mistake and the capacity of the team handling exceptions.

This is different from buying the model with the highest average accuracy. A model can be useful when it handles a well-defined subset reliably and recognises when to defer. It can be expensive when it confidently makes the wrong decision on the cases that matter most.

An October 22 preprint on cost-aware ticket classification examines that trade-off. The practical deliverable for your business is a decision policy with explicit costs, held-out evidence and a review queue that can absorb the cases sent to it. Research paper

Begin with one action the system is allowed to take. “Automate support” is too broad. “Assign an incoming request to one of eight internal queues” is specific enough to label, evaluate and observe after deployment.

Then describe what happens when the action is wrong. A misrouted low-priority request may need a simple reassignment. A time-sensitive request sent to the wrong team may lose hours before anyone notices. The same count of errors can conceal very different operating consequences.

Separate the classification from any subsequent action. A useful prediction does not automatically justify sending an external message, modifying an account or closing a case. Those actions can require their own checks and authorization, even if the initial route is clear.

Write down the available dispositions: automatic routing, human review, request for missing information and an explicitly handled unknown category. Forcing every unfamiliar request into a known class can create confident errors that a threshold alone will not solve.

The corpus contains 47,837 tickets across eight classes. The TF-IDF plus logistic-regression baseline uses a 9,567-ticket test split; the prompted-LLM comparison uses a stratified 1,000-ticket subset. Study design

With correct/review/wrong costs of 0/1/5, the baseline's cost falls from 0.763 to 0.472 per ticket: approximately 38.1%, in assigned cost units rather than dollars. Its calibration-selected raw-score threshold is 0.60.

The fixed break-even policy instead defers every ticket in seven of eight prompted-LLM configurations. Abstention prevents those weak confidence signals becoming automatic errors; it does not repair the classifiers. The baseline result therefore does not establish commercially useful LLM routing. Results and discussion

The transferable idea is to compare decision policies using an explicit loss model. Your ticket mix, error consequences, review time and classification system may differ from the study's. A useful replication would retain the method while replacing the assumptions with ones the operational owner can defend.

Consider a simplified, illustrative decision model. Let p be a calibrated probability that the proposed automatic decision is correct. Let Cw be the cost of a wrong automatic decision and Ch the cost of human review. Assume correct automation has zero incremental cost and reviewed cases are resolved correctly.

Under those assumptions, the expected automatic cost is (1 − p) × Cw. Review costs Ch. Automatic handling is cheaper when:

(1 − p) × Cw < Ch, or p > 1 − Ch / Cw.

Using hypothetical costs of one unit for review and five for a wrong decision gives p > 0.80. At exactly 0.80, the two options have equal expected cost in this simplified model; the workflow needs a tie policy.

This derived 0.80 probability threshold is not the paper's 0.60 raw-score threshold. One comes from an idealised expected-cost equation; the other is a selected cutoff for a specific model's scores and data.

The distinction matters because a raw score of 0.80 need not mean 80% correctness. The equation is only useful when its input and cost assumptions match what the workflow can actually estimate.

A model can express high confidence and still be wrong. Asking it for a number does not establish that cases receiving 0.90 confidence will be correct approximately 90% of the time. That relationship needs evidence from labelled examples resembling the intended workload.

For triage, the relevant probability is that the proposed route is correct. Check how observed correctness changes across score bands. Keep the number of examples in each band visible: a band containing only a few cases offers weak evidence, however attractive its observed accuracy.

Use separate data for fitting any calibration mapping and for evaluating the final policy. If the same examples determine the mapping, select the threshold and demonstrate its success, the reported performance can be optimistic. Preserve a later test period or another genuinely held-out set.

Calibration and ranking answer different questions. Calibration asks whether probabilities correspond to observed frequencies. Ranking asks whether the score places more reliable decisions above less reliable ones. A system that assigns the same score to every request offers little help choosing which individual requests to defer, even if that single score matches overall accuracy.

The distinction is also explained in the scikit-learn probability calibration documentation, checked on November 10. The software documentation is a current explanation of the method, not evidence of a November 8 product launch or a result from your workflow.

A single wrong-decision cost is useful for explaining the arithmetic, but many workflows need more detail. Confusing two similar internal teams may be inexpensive; missing a request with a deadline may be costly. Treat those as different errors when the distinction changes the decision.

Build a small cost table with the operational owner. Include the direct correction work, the cost of delay and any consequence that cannot be represented reasonably as a routine average. If a class must always receive specialist review, state that as a rule rather than disguising it as an ordinary monetary preference.

The following example uses hypothetical units solely to show how the threshold changes. It assumes review costs one unit and resolves the case correctly, as in the earlier equation.

Do not copy these costs into a production policy. Estimate your own consequences and test whether the decision changes across a reasonable range. If a small change in an uncertain cost estimate reverses the decision, present that sensitivity instead of claiming a precise optimum.

For more complex classifications, the cost depends on both the predicted route and the true route. Evaluating that full error pattern can be more useful than asking which model has the best overall accuracy. It exposes where a seemingly strong average hides a concentrated operational problem.

The simple threshold assumed perfect review at a fixed price. Actual review has handling time, queue delay and a residual chance of error. A person may also need missing information or specialist advice before making a decision.

Extend the comparison when those effects matter. One illustrative formulation is review handling cost + expected review error cost + expected delay cost. Compare that with the expected cost of the automatic action, including any relevant processing cost. Avoid counting the same delay or correction work twice.

Capacity places a further limit on the policy. If 1,000 requests arrive each day and a chosen threshold sends 30% for review, the team receives 300 review cases. If it can complete only 200, the review backlog grows by 100 per day under these hypothetical steady conditions.

Lowering the threshold merely to hide that backlog transfers work into the automatic path and can increase errors. Better options may include narrowing the automated scope, improving input quality, adding review capacity or changing the service commitment. Evaluate the consequences openly.

Our article on model costs and human review work explains how to measure inspection and correction effort. That ledger supplies useful inputs here; the abstention policy determines which cases enter it.

Compare at least three policies on the same labelled evaluation set: automatic decisions for all eligible cases, human review for all eligible cases and the proposed selective policy. Use the same cost assumptions and preserve the original case distribution.

Report more than a single expected-cost number. Show the automatic coverage, error rate among automatically handled cases, review volume and error distribution by important class. A policy can lower average cost while still failing on a category that needs special treatment.

Include uncertainty where the sample is small. If there are very few examples of an expensive error, observing none does not prove that the error cannot occur. A targeted supplement can help investigate that failure mode, but keep it separate from the representative sample used to estimate routine volume.

Record the selected threshold and freeze it before the held-out evaluation. If the result reveals a problem and you revise the policy, that is legitimate development; evaluate the revision on fresh evidence rather than presenting the reused test set as untouched.

For LLM-based routing, account for the full execution path. Classification prompts, retries and additional checks can add cost. Our model-router overhead guide explains why the routing step itself belongs in the ledger.

Begin by observing the proposed policy while people continue to make the decisions. Compare its suggested automatic actions with reviewed outcomes and check whether the anticipated review volume is plausible. Historical labels may omit details that become obvious in a live workflow.

When the evidence supports automatic handling, enable it for a bounded category with an explicit recovery path. Retain a sample of automatically routed cases for later review; inspecting only deferred cases would hide errors in the part of the system allowed to act.

Track changes in input mix, score distribution, review age and corrected routes. A new product, new customer group or revised queue structure can make yesterday's calibration less informative. Define which changes require another evaluation before they occur.

Make corrections useful. Record whether the failure came from ambiguous input, a wrong label, insufficient context, a changed taxonomy or a poor decision despite adequate information. Different causes need different repairs; retraining is not the answer to every operational ambiguity.

Assess the policy by cost per successful task, with the success criterion fixed. Lower review volume is valuable only if the completed decisions still meet that criterion and the remaining review queue remains workable.