Reasoning Effort Cost vs Quality Benchmarks

Frontier models in 2026 ship a reasoning_effort dial. The dial works — quality lifts 8 to 22 points across the curve. The dial also costs — fees inflate 4-17×, latency 5-60×. The economic question is no longer which model; it is which tier, picked per workload.

We ran 900 tasks across five frontier models and three effort tiers on math (AIME 2026 problems), code (Expert-SWE refactor), and analytic reasoning (GPQA Diamond). The crossover point — where higher effort starts costing more per correct answer than the quality lift earns — is task-specific and lives at different tiers for each workload. This piece publishes the data and the decision matrix.

Cost-per-correct-answer is the right unit. A 22-point pass-rate lift at 17× cost is a great deal on a hard math contest where the answer is binary; the same lift on a PR-scale review where humans edit anyway is a waste. The matrix in §07 maps nine common workflows to the right tier.

Five frontier models (GPT-5.5 Pro, Claude Opus 4.7, Gemini 3 Pro Deep Think, Grok 4.5 Reasoning, DeepSeek V4) tested at three reasoning_effort tiers: low, medium, high. Each provider exposes the dial differently — OpenAI uses the explicit reasoning_effort parameter; Anthropic uses extended thinking budget; Google Deep Think uses thinking_budget; xAI Grok uses reasoning_mode; DeepSeek uses an internal CoT toggle. We normalised by approximate token-spend tier rather than vendor parameter name.

Three task families, 60 problems each, run three times per model+effort cell — 900 task runs total. Pass-rate computed as majority vote across the three runs.

Math is where reasoning_effort earns its keep. Across all five models, the low-to-high tier delta on AIME 2026 is 18-22 points. The chart below shows the per-tier pass-rate for each model.

Two reads matter. First: the low-to-high curve is steeper on math than on any other family — 22 points on GPT-5.5 Pro, 18-22 across the board. The compute pays for itself in verifiable correctness. Second: DeepSeek V4 at high reasoning (84.2%) beats GPT-5.5 Pro at low (69.3%) and is competitive with all four frontier closed-source models at medium. The cost gap (15-30×) is substantial.

Code reasoning behaves differently than math. The marginal lift from medium to high is small (3-5 points across the frontier) and sometimes negative — extra reasoning time spent on Expert-SWE refactor often introduces over-engineered solutions that fail integration tests. Medium is the right default for production code workflows.

Graduate-level scientific reasoning sits between math and code on the curve shape. Quality lifts cleanly from low to medium (12-15 points) and continues to lift modestly from medium to high (3-7 points). The medium-to-high band is where most analytic-reasoning workflows should sit, picking the tier by latency budget.

Quality and cost in isolation tell you nothing. The chart that matters is cost-per-correct-answer — total spend on a task family, divided by the number of correct answers. Below: cost-per-correct for AIME 2026 across the model+effort grid.

The ranking inverts. GPT-5.5 Pro at high effort wins on raw pass-rate (91.7%) but lands at $0.78/answer — 19× the DeepSeek V4 high-effort cost ($0.04). For workloads where the 7.5 percentage points of extra correctness do not justify the cost (most internal workflows), DeepSeek V4 at high reasoning is the procurement floor.

Cost and quality are two axes; latency is the third. Reasoning modes inflate TTFT 5-60× depending on model and tier. For chat UX workflows with sub-2-second latency budgets, high reasoning is unusable regardless of capability ceiling.

The matrix below maps nine workloads to the right effort tier based on the empirical pass-rate curves and cost-per-correct numbers. Use this as the starting policy, then measure against your specific quality bar.