ARC Prize Verified Opus 5. That Is Rarer Than It Sounds.

ARC Prize published independent verification results for Claude Opus 5 on July 24, 2026: 30.16% on ARC-AGI-3 at High reasoning effort — the highest score recorded on that benchmark to date. The number matters. But the process behind it matters more, because it is one of the very few launch-window benchmark results that a third party administered rather than the vendor itself.

The same 48 hours produced a second ARC-AGI-3 headline from a different source: Anthropic’s own launch page, which claims a result “three times as high as the next-best model” — with no methodology, no harness description, and no mention of ARC Prize anywhere on the page. Same benchmark, same week, two unlinked claims with very different evidentiary weight.

This guide uses that split-screen moment as a teaching case. We cover exactly what ARC Prize verified and how, what even ARC Prize does not disclose, the independent cross-checks that complicate the headline, and a practical scorecard for reading the next benchmark claim that lands in your feed.

The ARC Prize results page for Claude Opus 5, published July 24, 2026, covers all three generations of the ARC-AGI benchmark family. On ARC-AGI-1’s 400-task Public Eval set, Opus 5 scored 97.5% at both High and Max reasoning effort — the two settings produced identical results on this benchmark. On ARC-AGI-2’s 120-task Public Eval set, it scored 90.4% at Max and 88.3% at High. And on the ARC-AGI-3 Public Demo set of 25 interactive environments, it scored 30.16% at High effort — rounded to 30.2% in ARC Prize’s own social posts, a secondary channel we could only corroborate through search snippets, and the highest score recorded on that benchmark to date.

The run also cleared new ground. In ARC Prize’s words, Opus 5 “completed five additional Public Demo environments that no model had previously beaten, demonstrating strong logical reasoning.” Five environments out of a 25-environment set moving from never-solved to solved in a single evaluation is a meaningful capability signal — and it is a signal you can trust more than usual, precisely because the organization reporting it had no stake in the model’s launch week.

Note what the chart does not show: a Max-effort bar for ARC-AGI-3. That absence is deliberate, disclosed, and — as we will argue in Section 04 — one of the most instructive details on the whole page.

The ARC-AGI family’s stated design philosophy is “Easy for Humans, Hard for AI.” The methodology page targets fluid intelligence — reasoning over genuinely novel problems — rather than crystallized intelligence, the recall of knowledge absorbed during pretraining. Tasks are restricted to “core knowledge priors”: cognitive building blocks present at birth or acquired very early in human development. The point of that restriction is anti-contamination by construction — a massive pretraining corpus should confer no hidden advantage.

The three generations escalate that philosophy. ARC-AGI-3 is the sharpest break: instead of static input/output puzzles, agents are dropped into live interactive environments where they must, per ARC Prize, “explore novel environments, acquire goals on the fly, build adaptable world models, and learn continuously.” The benchmark explicitly measures “skill-acquisition efficiency over time” and “long-horizon planning with sparse feedback” — not just final-answer accuracy. ARC Prize, Inc. administers the benchmark with a public leaderboard, and participation also runs through an ARC Prize 2026 Track on Kaggle.

That design lineage is why a 30.16% reads as a record rather than a failure. Static benchmarks that frontier models saturate stop discriminating between them; an interactive benchmark where the best recorded score is under a third of the ceiling still has years of signal left. For a deeper treatment of contamination and saturation dynamics, see our guide on how to read a benchmark leaderboard.

Here is the detail that makes this launch a teaching case. Anthropic’s own Opus 5 launch page, published the same day as ARC Prize’s results, cites an ARC-AGI-3 result of its own: “Opus 5’s score is three times as high as the next-best model.” That sentence carries no methodology, no harness description, and no third-party name — the page never references ARC Prize at all. The vendor-framed headline and the independently verified number arrived from two different, unlinked sources within the same 48 hours.

The rest of Anthropic’s launch-day benchmark story is fully vendor-run: results on Frontier-Bench v0.1 (Anthropic’s own eval), CursorBench 3.2, OSWorld 2.0, and Zapier AutomationBench were all produced by the vendor, on harnesses and effort settings the vendor selected. We covered those numbers — and the pricing story around them — in our Opus 5 launch analysis, so we will not re-derive them here. The point is the category difference: those are claims a buyer must take on trust; the ARC number is a claim a buyer can trace to an administrator with no launch to sell.

This is the subtle version of a common failure: even a credible administrator has informal channels, and numbers that appear only in social posts do not carry the same weight as numbers on the results page. A rigorous reader keeps three tiers in mind — primary page, administrator’s social channel, third-hand press — and lets a claim’s tier set its confidence, not its virality.

The most quietly instructive line on the results page is a limitation, stated without spin.

Max reasoning effort — the setting that lifted ARC-AGI-2 from 88.3% to 90.4% — was simply not run on ARC-AGI-3. So the single most-quoted number of the launch window is, by its own administrator’s admission, potentially an undercount of the model’s ceiling. That kind of caveat is what honest benchmarking looks like: publish what was tested, name what was not, and resist extrapolating. Effort settings are not a footnote, either — High versus Max is exactly the cost-quality lever we quantified in our reasoning-effort benchmarks, and it can move scores by whole percentage points.

But the page’s transparency has edges, and naming them matters for calibration. The results page discloses no attempt limit per environment, no harness or scaffold description, no cost-per-task figure for the run, and no named auditor credential beyond ARC Prize itself. On cost, the page offers only a qualitative framing for Max-effort runs elsewhere: “This is competitive with previous frontier leaders, though at slightly higher cost.” No dollar figure for the ARC-AGI-3 run exists in any source we checked — so you will not find one in this post either.

The honest conclusion: third-party verification is a spectrum. An independent administrator with partial methodology disclosure sits well above a vendor’s own harness — and still short of fully open methodology. Both facts fit in one head.

A single benchmark — even a well-administered one — is one measurement, not a verdict. The most useful counter-evidence this week comes from an independent eval that is methodologically adjacent to ARC-AGI-3: Guanghan Ning’s “Witness” benchmark, an interactive puzzle-game evaluation administered separately. As reported by The Decoder, Opus 5 scored 43.4 there — described as “statistically equivalent to competitors.”

The Decoder’s framing of that gap is worth quoting because it is the properly cautious read: the ARC-AGI-3 jump “may reflect targeted optimization rather than broad reasoning advances,” while “the evidence remains inconclusive.” Two interactive benchmarks, one dramatic outperformance, one statistical tie. That does not invalidate the ARC result — it bounds what the ARC result can be said to prove. A record on one benchmark is evidence of capability on that benchmark’s task distribution, not a general-intelligence coronation.

This is our core interpretive claim, and it cuts both ways: the same skepticism applies to any model that tops any single leaderboard this year, including ones we like. The practical discipline is to require two independent measurements before updating a routing decision — and to weight the measurement whose administrator, harness, and sample size you can actually name.

The third administration model — crowd preference — has its own failure mode, visible live this week. On the Arena.ai text leaderboard as of this week, several current frontier and near-frontier entries carry an explicit “Preliminary” tag with wide error bands tied directly to low vote counts: Meta’s muse-spark-1.1 at 1495±7 Elo on 7,927 votes, Moonshot’s kimi-k3 at 1486±10 on 3,619 votes, Google’s gemini-3.6-flash at 1485±9 on 4,747 votes, and Alibaba’s qwen3.7-max-preview at 1475±10 on 3,714 votes. Long-established entries carry no tag and vote counts an order of magnitude higher — grok-4.20-beta sits on 60,330 votes with a ±4 band.

The pattern has a mechanism: early votes on a newly listed model skew toward fans and early adopters, and rankings visibly move once volume catches up. A dated example from the same window: Moonshot’s Kimi K3 ranked first on the Arena WebDev leaderboard with a “preliminary score of 1,679,” the source explicitly noting “Preliminary result based on 1,757 votes and likely to change” — a nine-day-old snapshot we cite purely as a methodology illustration, since the K3 release itself is covered separately. The lesson generalizes: a crowd eval’s sample size is part of the score, and any number quoted without its vote count is missing a significant digit.

Put every claim from this launch window side by side and the pattern teaches itself. The table below grades each published number on the five disclosure dimensions a buyer can actually check: who administered it, whether the harness is described on the public page, whether an attempt limit is disclosed, whether the sample size is shown, and whether the score is final or explicitly preliminary. Every cell reflects what the cited public page disclosed as of July 26, 2026 — except the Kimi K3 · Arena WebDev row, which reflects the dated July 17 snapshot referenced in Section 06.

Two readings jump out. First, no row scores clean across all five columns — not even ARC Prize. Second, the failure modes differ by administration model: third parties tend to under-disclose harness detail, vendors tend to under-disclose everything except the win, and crowd platforms disclose sample size honestly but cannot escape early-vote skew. Knowing which failure mode you are looking at is most of the skill.

Distill the launch window into a repeatable protocol and you get five questions. They would have correctly triaged every number discussed in this post.

The fifth question is the cross-check: does a second, independent measurement agree? The Witness-benchmark tie next to the ARC record is this week’s live demonstration that one leaderboard is never enough. The same verification instinct applies beyond benchmarks — our buyer’s scorecard for agent claims runs the identical playbook against “agentic” marketing.

Looking forward, we expect the verification gap to become a competitive axis in its own right. As launch cadence accelerates, vendors that invite third-party administration early will increasingly earn a trust premium over vendors that publish only their own harnesses, because buyers are learning to price the difference. If your team is choosing models for production and wants evals run on your workloads rather than anyone’s marketing page, that comparative benchmarking is exactly where our AI transformation engagements start.