When Agent Tables Contradict Their Own Stated Formulas

Derived-cell defects in agent-written tables are the errors your fact-checker is structurally blind to: a table states its scoring formula, cites real inputs, and prints an output the formula does not produce. Nothing in the row is fabricated. Each price, count, and score traces to a source. The failure lives entirely in the arithmetic connecting them — and arithmetic is the one thing a source check never touches.

The stakes rise with volume. Teams now ship agent-written pricing tables, comparison matrices, and scored bar charts at a pace no human editor recomputes by hand. Every one of those artifacts carries derived values — totals, percentage deltas, min–max ranges, stated counts — and every derived value is a place where correct inputs can silently combine into a wrong output that reads as authoritative.

This framework defines the defect class, documents three first-party catches from a single day of adversarial QA on our own pipeline, connects the class to thirty years of spreadsheet-error research, explains the model-side and schema-side mechanisms that produce and conceal it, and specifies a four-subtype taxonomy with the recompute gate that catches each one.

Most coverage of AI content errors talks about hallucination — invented facts, fabricated citations, confident claims with no source behind them. The derived-cell defect is different in a way that matters operationally. In this class, every individual value in the artifact is real and independently verifiable. The plan prices are the vendor’s actual prices. The lab names are actual labs. The raw scores trace to actual measurements. What fails is the derivation: the range that should bracket those prices doesn’t, the count that should match that list doesn’t, the formula that should produce those bars produces different numbers.

That property is precisely what makes the class dangerous. A reviewer who checks each number against its source finds nothing wrong, because each number is right. The wrongness only appears when someone recomputes the relationship between numbers — and recomputation is not a step in any conventional editorial fact-check. Hallucination rates are usually measured on facts, not arithmetic; our review of hallucination-rate benchmarks covers that adjacent, better-studied failure mode, and almost none of it transfers to this one.

This is not a theoretical taxonomy. On 2026-08-23, an internal adversarial QA pass on our own agent-written content caught this exact defect class three separate times in a single day — three different artifacts, three different subtypes, one common signature. We publish the full measured record of that pipeline’s verify-and-fix telemetry in our agentic content pipeline yield ledger; here we use only the three instances, described as a class, with no names attached.

The common signature is the tell: all three instances passed a first-pass fact-check, because every individual number involved — each plan price, each lab name, each raw score — was independently real and sourced. The defect lived entirely in the arithmetic connecting them, not in any single fact. They were caught only when a second, deliberately skeptical pass recomputed derived values from their stated inputs instead of checking inputs against sources.

One dated data point does not make a rate, and we won’t manufacture one. What three same-day catches do establish is that the class is live in a real production pipeline, that it recurs across distinct artifact types (prose ranges, summary cards, bar charts), and that it clusters into recognizable subtypes — which is exactly what a defect taxonomy and a targeted gate need.

The derived-cell defect did not arrive with language models. It is the signature failure mode of the spreadsheet era, documented by a research literature that AI-content commentary almost never cites. Field audits of operational spreadsheets conducted between 1995 and 2004 — reviewed in a meta-analysis of 13 field-audit studies spanning 43 audited spreadsheets — found an average cell error rate of 5.2%, and that 94% of the audited spreadsheets contained at least one error. Later, larger work found lower but still material rates: a 2007 study by Powell and colleagues across 270,722 formulas measured a 1.79% cell error rate, falling to 0.87% when counting only wrong-result errors, per the Tuck critical review of the spreadsheet-error literature. Laboratory development studies — 14 studies, 967 participants — averaged 3.9%. The human-factors observation cited alongside this work treats a 2–5% error rate in complex cognitive tasks as a rough directional baseline, not a precise benchmark.

The consequences are equally documented. The EuSpRIG horror-stories archive has cataloged real spreadsheet-error incidents since 1995. A Canadian power company took a $24 million loss in 2003 after a cut-and-paste error mispriced a bid. Norway’s sovereign wealth fund disclosed a loss of roughly NKr 980 million (about $92 million) traced to an incorrect date entered into a benchmark calculation — a single bad input propagating through a derived calculation undetected. And the Reinhart–Rogoff austerity-research spreadsheet remains the canonical academic example: independent researchers who obtained the underlying file found coding errors, selective exclusion of available data, and unconventional weighting — an error class that persisted through peer review and years of public citation until someone recomputed the results from the raw inputs. The lesson is precedent, not analogy: derivation errors survive review because the inputs look right.

This is the interpretive point most AI-content commentary misses: the derived-cell defect is the same class human spreadsheet authors have produced for thirty years, now authored by an agent instead of an analyst. The spreadsheet literature also settled what fixes it — audit disciplines that recompute cells from inputs, not reviews that admire the layout. That settled answer transfers to agent-written content almost unchanged.

Why would a system that can state a formula fail to apply it? The clearest dated evidence comes from Apple’s GSM-Symbolic study (published October 7, 2024), which tested 25 contemporary models of that generation — GPT-4o-class systems, Llama3, Phi-3, o1-preview among them — on symbolic variants of GSM8K math word problems. The results, per the paper: 21 of the 25 models tested scored below the right tail of their own GSM8K performance distribution when given symbolically varied versions of the same problem template, and Phi-3-mini suffered up to a 65% performance drop when irrelevant-but-plausible clauses were added to otherwise unchanged problems. Performance was not stable across superficially equivalent restatements of the same arithmetic.

Two cautions bound that evidence. First, those percentages belong to 2024-era models and should not be projected onto current systems: no dated, citable benchmark of 2026-era frontier models on multi-step arithmetic-in-tables tasks surfaced in our research for this piece, so every current-model claim here stays qualitative. What can be said qualitatively is that current frontier models still generate plausible-looking but unverified arithmetic by default — generation predicts tokens; it does not execute formulas. Second, the mechanism the GSM-Symbolic authors described — pattern-matching that is highly sensitive and fragile to input changes, deteriorating as problem clauses accumulate — maps directly onto table generation. GSM8K problems chain roughly 2–8 arithmetic steps; a scored table chains the same formula-to-intermediate-to-cell structure across every row. A more recent diagnostic line of work (arXiv:2605.00817) narrows the question further to arithmetic procedural execution — whether a model faithfully carries out an explicitly specified step-by-step procedure — which is the agent-relevant framing: Instance C above is precisely a procedural-execution failure, applied unevenly across rows.

The most common rebuttal — “we use structured outputs, our tables are validated” — is the second half of the mechanism. Anthropic’s native Structured Outputs feature (constrained decoding via an output_format JSON-schema parameter) launched in public beta on November 14, 2025 and reached general availability for the Claude 4.5-generation models. Anthropic’s own framing is exact: the model literally cannot emit tokens that violate the schema — and it can still emit a perfectly schema-valid, arithmetically wrong value. A well-formed table whose cells contradict their own stated formula is schema-valid by construction.

Independent practitioner analysis converges on the same framing: structured outputs are a transport guarantee, not a correctness guarantee. The distinction holds across hosting surfaces — Amazon Bedrock’s equivalent Structured Outputs support for Claude 4.5 models reached general availability on February 4, 2026 with the same shape-not-values contract — so this is a property of constrained decoding itself, not one vendor’s caveat.

Naming subtypes matters because each one has a distinct, cheap, mechanical check. Three of the four below come directly from our 2026-08-23 first-party catches; the fourth — a total that does not equal the sum of its rows — is the classic finding of the spreadsheet-audit literature, included as the predictable fourth variant of the same class. Together they define what a recompute gate actually executes.

Note what unifies the four gates: none of them consults a source, and none of them requires judgment. Each is a deterministic computation over values already present in the artifact. That is what makes the class fully automatable to catch — and what makes shipping it inexcusable once you have named it.

The full pattern operates at two points in the pipeline. At generation time, derived numbers should be computed, not generated: route arithmetic through an execution tool so the model plans while an interpreter calculates. Anthropic’s code execution tool runs Python or Bash in a sandboxed container within a single API conversation, explicitly for tasks like performing complex calculations and analyzing data sets — a division of labor where the model interprets and the sandbox executes. The broader function-calling literature converges on the same principle: calling a calculator tool with operands, rather than generating the result as free text, separates language understanding from precise computation.

At publish time, the gate assumes generation failed anyway. A separate verification pass — ideally a different agent with a deliberately skeptical brief — executes the four checks from the taxonomy against every table, card, and chart: re-run each stated formula against each row, sum every total, count every enumeration, confirm every range brackets its members. The output is a diff, not an impression. This is the same audit discipline we apply when agents audit a costing engine, where derived-cell defects hit hardest financially, and it is the pattern we install in client pipelines through our AI transformation engagements and operate daily inside our own content engine.

Looking forward, the trend line is not subtle. Agent-written artifacts with derived values are compounding across every content, pricing, and reporting surface, while the defect class scales with volume rather than with model quality — a structural property of generation, as Section 04 argues, not a bug any single release patches away. Our projection: within a few years, a recompute gate will sit in content pipelines the way CI sits in code pipelines — an unremarkable, mandatory check that nobody ships without — and the teams who adopt it early will be the ones whose published numbers survive the audits their competitors fail. The spreadsheet era needed a decade and a horror-story archive to institutionalize auditing; agent pipelines can skip straight to the discipline.