Which AI Model Actually Writes Best? The Boards Disagree

If you have just read a “best AI for writing” article, the honest answer to whether you should believe it is: not on the strength of either leaderboard alone. EQ-Bench Creative Writing v3 and arena.ai’s Creative Writing category — whose page states an August 27, 2026 snapshot — do not merely re-rank the same models. They invert. claude-fable-5 is first on arena.ai and sixth on EQ-Bench; kimi-k3 is second on EQ-Bench and twenty-third on arena.ai.

The disagreement is measurable, and — more usefully — explainable. One board is a human blind-voting arena; the other is an LLM-judged rubric-plus-Elo pipeline. The full side-by-side table is directly below, and the rest of the page explains why two carefully built instruments pointed at the same task return opposite answers.

Two definitions before the table, because both carry weight in it. Creative Writing is the narrow category both boards publish: fiction, storytelling, and poetry written to a prompt. Leaderboard platforms track it separately from general “Writing” tasks like emails, essays, and editing, so a rank here is a creative-writing rank, nothing broader. An Elo score, as leaderboards use the term, is a relative rating inferred from many pairwise comparisons: winning against a strong opponent moves a model up more than winning against a weak one. It is meaningful only within one board’s pool, solver, and anchor points — which is why the two Elo columns below run on visibly different number lines and must be read as two separate rankings, never one.

Read the first three rows again. The model arena.ai’s voters rank first sits 183 Elo behind the EQ-Bench leader on EQ-Bench’s own scale. The model EQ-Bench ranks second is twenty-third with human voters. The model EQ-Bench ranks first does not crack arena.ai’s top nine. This is not noise around a shared consensus — the two boards return structurally different orderings of the same vendor lineups on the same nominal task. Neither board is the writing leaderboard, and any article that cites exactly one of them as settled truth has made a choice it probably has not told you about.

This is a comparison of two published datasets, not a benchmark we ran. That makes the matching rules — which names were paired, which scales were kept apart, which board stamps its own snapshot — the part a citing reader needs most.

First, EQ-Bench Creative Writing v3’s top ten. Note the rubric column alongside the Elo — the same board publishes two scores per model, and section 05 is about the rows where they tell different stories. Length is the average output length the board reports per model, and it does real analytical work here.

Now the arena.ai rows for the same families, from a page that states its own snapshot date — Aug 27, 2026 — and its own sample: 1,214,472 human votes across 393 models in this category. The confidence intervals matter: glm-5.3-max, qwen3.8-max, muse-spark, and kimi-k3-max sit within a few points of one another, with overlapping intervals, on 1,286 to 3,208 votes each.

One row needs a correction the board has not made. arena.ai’s licence label reads “Z.ai · MIT” for glm-5.3-max. The Hugging Face model card for zai-org/GLM-5.3 states license: other with license_name: glm-5.3 — a bespoke licence, not MIT. We documented that licence in detail in our August 28 post on the GLM-5.3 weights. To be precise about what was checked: the correction rests on the zai-org/GLM-5.3 model card, and glm-5.3-max may be a hosted-only tier — but a filter column that lets a reader sort by “MIT” and returns this model is disagreeing with the model’s own card. The two boards, in other words, diverge on a checkable fact as well as on scores.

The inversion stops being mysterious once you look at how each number is produced. arena.ai is human pairwise voting: two anonymous outputs shown blind, side by side, one vote for the better one, aggregated through a Bradley-Terry model — the standard statistical method for turning many pairwise win-loss records into one rating per model. EQ-Bench is LLM-judged, twice over: outputs are first graded against a scoring rubric by one Claude judge, then run through pairwise matchups against neighbouring models — judged by a second, different Claude judge, with win margins feeding a modified Glicko/Trueskill-style solver — a rating system of the same broad family as Elo, extended to weigh how decisively each matchup was won — run until ranks stabilise. Per the site’s March 1, 2026 note, the Elo judge is Claude Sonnet 4.6 while the rubric judge remains Claude Sonnet 4: two judges, two columns, one leaderboard.

Two design details deserve their own sentences. First, EQ-Bench is adversarial by design — its prompts are chosen to be “challenging for weaker models and therefore highly discriminative”, and the site says plainly that “the purpose of the evaluation is not to help models write their best. Instead, we are deliberately exposing weaknesses.” arena.ai measures preference under whatever its voters bring. An instrument built to expose weaknesses and an instrument built to aggregate preference are answering different questions even when the category name matches. Second, arena.ai’s Style Control exists precisely because human votes carry style bias: LMSYS’s own launch post explains, “We explicitly model style as an independent variable in our Bradley-Terry regression. For example, we added length as a feature — just like each model, the length difference has its own Arena Score!” The toggle is live on the Creative Writing page today; what its activation does numerically to the 2026 gaps in this post could not be captured, so this page claims no narrowing or widening from it.

You do not need two leaderboards to see the disagreement — EQ-Bench publishes it in two columns of its own table. gpt-5.5 scores 85.05 on the rubric against claude-opus-5’s 85.35 — a 0.3 gap on graded quality — while sitting 272 Elo lower in the pairwise ranking, at 2.2× the output length. Further down the same table, horizon-beta posts a rubric of 83.30 — level with claude-opus-4-8 — at 14,202 characters, with an Elo of 1624.5. The board publishes both columns and they do not tell the same story.

One more pattern from the same table, because it breaks a default assumption buyers carry: newer is not better at writing. On EQ-Bench, claude-opus-4-8 (1835.3) ranks below claude-opus-4-7 (1906.4), and claude-sonnet-5 (1787.6) ranks below claude-sonnet-4-6 (1804.5) — same-vendor version regressions, on this benchmark, published in the vendor-neutral place vendors do not control. If a leaderboard only ever confirmed release-note ordering, it would not be measuring anything.

Here is the finding that survives everything above. The two boards disagree about nearly every individual rank — and still agree about the shape of the market: on each board, measured on its own scale, the highest-ranked open-weight model in the rows above sits a few dozen points behind that board’s #1 — 38 points on arena.ai, 45.3 Elo on EQ-Bench. These are two separate observations on two incompatible scales, and they concern different models. That is what makes the agreement striking rather than circular.

Neither board states this finding — each publishes only its own ranking, so the convergence only becomes visible when the two are laid side by side. It is also the most defensible takeaway for a buyer: whichever instrument you trust, the gap between the top open-weight models on these boards and the frontier is real but narrow, and the models occupying it are cheap to try.

So which board should you act on? Neither, on its own — and this post will not hand you a winner, because a single verdict is exactly the artefact the data argues against. We have published a head-to-head frontier-model verdict where the evidence supported one; here the evidence supports a refusal. What the two boards give you instead is a pair of directional signals with known, documented biases — and EQ-Bench itself says the quiet part on its own methodology page: “The scores and rankings should only ever be interpreted as a rough guide of writing ability”, and “it’s good to be skeptical of benchmark numbers by default.”

For reading any leaderboard — contamination, category design, cherry-picking — our benchmark methodology guide covers the general literacy this post applies to one specific disagreement. If the model is only half the equation for you — the other half being prompts, style constraints, and editorial scaffolding — the free skills that measurably improve AI writing matter more than a five-place rank difference ever will. And if the real question is standing up an editorial pipeline where model choice, briefs, and quality control are designed together rather than argued from leaderboards, that is what our Content Engine service builds.