Decision Models: AI That Returns Probabilities, Not Text

A small new category of AI model answers questions without writing anything. You send it some text, such as a support message or an agent’s transcript, plus a list of typed questions: is this urgent, which team should get it, did the agent loop. It returns a probability for each answer and nothing else. On September 28, 2026, OpenRouter listed seven of these routes from four publishers. Three of the four publishers arrived in the previous four days.

For anyone running agents, the appeal is cost and speed. Output tokens are free, because there is no output text, and each answer takes a single pass through the model. The question is whether the answers are good enough to replace the language model many teams now use as a judge. The vendors say yes; the evidence so far is their own.

A normal language model writes an answer word by word, and if you want a label you parse it out of the text. A decision model skips the writing. According to OpenRouter’s route page, a request carries a state, which can be a string, an object or an array, and a set of questions. Each question has a type: noul for a yes/no probability, choice for a pick from options you define, or score for a position on an ordered rubric. The model returns the probabilities and your code acts on them.

These models do not run on the familiar chat endpoint. OpenRouter serves them through a separate, alpha Decisions API, and its page warns that chat-completions SDKs will not work with it. The format started with TypeSafe’s Jev, which we covered at its launch. Upstage describes Solar Decide as a System One endpoint, the name TypeSafe gave that format.

We filtered OpenRouter’s model catalog, read at 21:28 UTC on September 28, for routes whose output type is decisions. The listing dates are OpenRouter’s; the release dates are from each vendor’s own page where one exists.

The seventh route is ~typesafe/jev-latest, an alias that points at the current Jev version. As with any alias, pin a version for anything you measure. Solar Decide is the notable newcomer: it comes from an established model maker rather than a startup, and Upstage lists it as beta, with a 512K-token context that can take a whole document as the state.

Upstage describes Solar Decide’s probabilities as calibrated and says each decision is one forward pass. It publishes no benchmark, so there is nothing to compare yet.

Respan, which sells tools for tracing and evaluating AI agents, published two benchmarks with its September 24 announcement. Span-01 is built for one job: you write a plain description of a behaviour, such as user frustration or an agent misusing a tool, and it returns the probability that the behaviour is present, absent or not observable in a conversation. On the headline benchmark Respan ranks Span-01 first of nine models at 0.843 overall F1, ahead of GPT-5.6 Terra (0.837) and GPT-6 Luna (0.815); GPT-6 Sol is not in that set. Respan’s results by behaviour domain, which do include Sol, all run by Respan:

Two things stand out. Span-01 beats Jev, and Sonnet 5 (0.719 overall), in every domain shown. And GPT-6 Sol, a full frontier model, scores higher than all of them in six of Respan’s seven domains, by the widest margin on response quality; Span-01 leads only on privacy and secrets. Respan concedes the point in its own write-up. The pitch is not that a decision model is as good as the best judge, but that it is close enough at a fraction of the price.

Respan also ran 11 decision models through a test in which Span-01 itself did the judging, measuring accuracy, consistency, resistance to prompt injection and calibration. Jev 1.13 ranked first at 0.932 accuracy and Kev-4B scored 0.833. Treat that ranking with care: the judge was the vendor’s own model.

An illustrative workload shows why teams are interested. Suppose an agent platform checks one million conversations a month, each about 2,000 tokens long. That is two billion input tokens. The volume is invented for the example; the prices are the listed route prices.

• Span-01 at $0.02 per million input tokens: $40 a month, with no output charge.
• Solar Decide at the $0.05 route price: $100 a month.
• A Claude Sonnet 5.5 judge at $2 per million input tokens: $4,000 for input alone, before any output or thinking tokens.

The saving is real only if the cheaper model’s mistakes cost less than the difference. For many checks, such as routing a ticket or flagging a likely prompt injection for review, a small error rate is fine. For a decision that blocks a customer or triggers a refund, it may not be. Our post on model routers and classifier overhead covers the same trade-off for routing.

1. Label a few hundred real cases. Use conversations or tickets where you know the right answer. Vendor benchmarks do not use your definitions.
2. Set thresholds from that data. A probability is only useful with a cut-off, and the right cut-off depends on whether a false alarm or a miss costs more.
3. Send the uncertain middle to a stronger check. Respan’s own guidance is to pass borderline cases to a person or a frontier model. That keeps most of the saving and limits the damage from errors.
4. Rephrase your questions and re-run. A good decision model should give the same answer to the same question asked two ways. Respan measures this as a flip rate; measure it yourself.

If your agents already produce structured outputs, the structured output reliability guide covers the adjacent problem of getting typed answers out of a normal model.