Before You Cite an AI Research Claim: A Seven-Step Check

On October 6, 2026 OpenAI published "Sharing AI progress in mathematics" and opened a public GitHub repository containing 722 manuscripts, grouped into 372 families of results, all produced by an internal model OpenAI has not released. The repository's own README says the collection holds "results at different stages of verification", that not all have a formal proof attached, and that "some of the unformalized results could have issues". The temptation is to summarise that as AI solving open problems. Some such summaries will turn out to be right. None of them are citable yet.

This post is for the editor, marketer or analyst who is about to write "AI has now proved" or "a model discovered" in a piece of their own. The release is the worked example, but the method is general. It applies to a lab's claim about drug candidates, to a vendor's claim that its agent "found" a security flaw, and to the next mathematics dump from whichever lab publishes one.

Two boundaries. We make no judgment about whether any of the 722 manuscripts is correct; that is for mathematicians, and on the release day none had been through peer review. And this is not a guide to checking the sources an AI research agent hands you, which our citation-checks reference covers. Here the AI is the author of the claim, not the assistant that fetched it.

Start with what the primary documents say, because the gap between them and the coverage is the whole subject. OpenAI's post announces "a broad range of new mathematical results produced by an internal frontier model" and points to the repository. It says OpenAI has been consulting the Advisory Group on Mathematics and Artificial Intelligence, hosted at the Institute for Advanced Study, and drew on the group's public recommendations to shape the release. It promises funding for workshops and conferences "around the understanding of major results produced by AI", and says the average result "used the equivalent compute of roughly three hours of ChatGPT Pro thinking".

The README adds the parts a careful citation needs. The model is internal and unreleased. The problem set was expanded after the model's scores on OpenAI's existing mathematics evaluations "saturated". Some manuscripts build on earlier output from the same models. Ten families come with abridged summaries of the model's reasoning. The README names two exceptions to the standard procedure, work on a zero-free region for the Riemann zeta function and a proof of the Hodge conjecture for CM abelian varieties, and says one write-up "was human edited for readability". Corrections will be recorded as new versions with the old ones kept accessible.

Those are the facts available on day one. Notice what is absent: no success rate, no list of problems the model failed, no statement that any human mathematician has read and understood a given proof, and no peer review. The absence is not an accusation. It is the shape of the evidence, and the seven steps below are built around it.

An AI research claim usually arrives in three layers: a press summary or social post, the lab's announcement, and the artefact itself. The artefact is the thing a sceptic could check: a manuscript, a dataset, a proof file, a benchmark log. For the October 6 release the artefact is the repository, and inside it the individual manuscript directory with its BibTeX block and, where it exists, its Lean file. The blog post is the announcement. Everything else is commentary.

Why it matters: each layer up drops a hedge. The README says "results at different stages of verification". The announcement says "new mathematical results". A social post sharing the announcement can easily say that OpenAI solved a named problem, and by the time the claim reaches a marketing deck it can read as AI now doing frontier mathematics, with no hedge left to lose. Citing the artefact pins your sentence to the layer that still carries the caveats.

The failure case is the citation that points at coverage of the thing instead of the thing. A link to a news article about the repository inherits that article's errors, and the article may have been written from the announcement rather than the README. If you cannot locate an artefact at all, the claim is a press release, and the honest wording is "the company announced", not "the company showed".

"Verified" is the most overloaded word in AI coverage, so unpack it. In mathematics there are three distinct kinds of check. Peer review: a journal sends the paper to experts who read it. Formal verification: the proof is rewritten in a language such as Lean so a computer can confirm each step follows from the last. Community replication: other mathematicians work through the argument, present it, build on it, and would notice if it fails. A claim can have one, two, all or none of these, and the wording you may use depends on which.

Lean is the one that gets misread, because "machine-checked" sounds final. What a Lean certificate establishes is that a particular formal statement follows from its stated assumptions. It does not establish that the formal statement is the theorem in the headline, that the assumptions are the standard ones, or that the proof is readable by a person. The advisory group's own recommendations allow for a proof to be "formalized modulo standard results that are accepted by the community", which is a legitimate status and also a status that must be stated. OpenAI's repository ships comparator challenge files so a reader can rerun a check; on release day it said many but not all manuscripts had been formalised.

The failure case is treating a formalisation count as a correctness count. A manuscript with no Lean file is not wrong; it is unchecked by that route. A manuscript with a Lean file is not "proved" in the headline sense until someone confirms the formal statement matches the claim. Write what the check covers.

Once you have the artefact, read the claim as the artefact states it, then compare it with the sentence you were about to write. The two usually differ in scope, in strength and in subject. Scope: the manuscript proves a case, the summary says the problem. Strength: the manuscript says "we obtain" with assumptions listed, the summary says "solved". Subject: the manuscript is one of 722 documents produced by a model, the summary says "OpenAI's AI".

The October 6 README demonstrates all three in one paragraph. It says the collection includes "results at different stages of verification" and that some results "build upon earlier results produced by the models". A reader who only sees "722 new results" misses that some results depend on others in the same release, so a gap in one could propagate. That dependency structure is exactly what a summary flattens and exactly what a citing writer needs.

The failure case is quoting the summary's verb and the artefact's number together, as in "OpenAI's model proved 722 theorems". The number is real and the verb is borrowed from a different document. Use the artefact's own verbs. If the artefact says "different stages of verification" or "could have issues", your sentence carries that hedge forward or it does not cite the artefact.

A pile of successes tells you nothing about a method until you know how many attempts produced it. This is the step the advisory group singled out in its fifth recommendation: when many results are released at once, the lab should publish a document that "explains how many other problems of comparable difficulty the models tried and failed to solve, as well as how the problems were chosen". OpenAI's README gives part of that: the model was posed approximately 4,000 problems. It does not give the rest. There is no count of failures, no account of how the problems were selected, and no success rate.

Resist the arithmetic. Dividing 722 manuscripts by 4,000 problems gives about 18%, and that number is wrong in both directions. Several manuscripts can come from one problem, because a family holds companion arguments and alternative proofs. One manuscript can draw on several problems. And OpenAI says the catalogue was filtered by "an appropriate level of significance", so the 4,000 may include problems whose results were set aside as minor rather than failed. A rate computed from those two numbers is a rate of documents per prompt, and nobody wants to cite that.

The failure case is the sentence "the model solved roughly one in five open problems it was given". It sounds like a finding and it is a division of two incommensurable counts. The defensible sentence is: OpenAI says it posed about 4,000 problems and published 722 manuscripts, and has not stated how many problems were solved or failed. That is less exciting and it is what the artefact supports.

On the day a claim lands, the expert response is mostly still being written. What you can check immediately is whether the experts set a standard beforehand, and whether the release meets it. For AI-generated mathematics that standard exists. The Advisory Group on Mathematics and Artificial Intelligence, nine mathematicians including Timothy Gowers, Martin Hairer and Edward Witten, hosted at the Institute for Advanced Study, published "Responsible Release of AI-Generated Mathematics" on September 29, 2026, after a survey that drew over 600 replies from the mathematical community.

Its opening position is not neutral. The group writes that labs are testing advanced problems on proprietary models "that remain inaccessible to the broader scientific community", and states: "we do not endorse this practice, and we ask them to stop testing advanced mathematical problems on proprietary models". The recommendations that follow are written for the case where labs do it anyway. OpenAI's post cites the same document as the guidance it drew on. So on release day a writer has two primary texts that disagree about whether the testing should have happened and agree about several things a release should contain.

The failure case is reporting the lab's statement that it consulted the experts as though it were the experts' endorsement. Consultation is not endorsement, and here the consulted group said in writing, a week earlier, that it does not endorse the practice. The table below sets the group's five release norms beside what the October 6 documents state, so the comparison is on the record rather than implied.

AI research artefacts change after release in a way journal papers do not. OpenAI's README says corrections and revisions "will be recorded as new versions, with previously released versions remaining accessible", and tells readers to cite the BibTeX block in each manuscript's directory. That is the right design, and it means a citation without a date or version points at a moving target. A manuscript you quote today may carry a correction notice next week; the sentence you wrote stays wherever you published it.

Why it matters for a general-audience writer: the half-life of "unverified" is short. Our July post on the Jacobian conjecture counterexample recorded a result that was checked informally by mathematicians within about a day and had no peer review at the time of writing. The honest wording then was that it had been checked informally and not yet peer reviewed. That sentence is still accurate as a dated statement and would be misleading as an undated one.

The failure case is the evergreen page that says "AI has proved X" with no date, written on the day of a release and never revisited. Put the date in the sentence or in a visible as-of line, cite the version, and treat the page as owing a revisit. If you maintain a correction policy, this is the class of claim it exists for; our correction policy for small publishers sets out the wording and the markup.

The last step is a routing decision. Given what the first six steps found, which sentence is true? The ladder runs from attribution, where you report that a party made a claim, through qualified description, where you report the claim and the check it has passed, to assertion, where you state the result as fact. Each rung needs a specific piece of evidence, and the evidence for the top rung is rarely available in the first week.

Attribution is not a cop-out. "OpenAI says its internal model produced a proof of the Unique Games conjecture, published October 6 with a Lean formalisation" is more informative than "AI proved the Unique Games conjecture", because it tells the reader who is claiming what and what has been checked. It is also the sentence that survives a correction: if the manuscript is withdrawn, the attributed sentence was still true when written. The assertion was not.

The failure case is choosing the rung by how the sentence sounds rather than by what the evidence supports. The router below maps the evidence state to the wording, so the choice is made once and applied consistently across a piece; it is the kind of rule we write into a client's content operations so it survives staff changes. For a fuller matrix of what formal proofs, measured results and demonstrations each establish, see our evidence-types reference.

One rule covers the whole ladder: a critic is quoted inside quotation marks with a name, and so is the lab. The advisory group's "we do not endorse this practice" is a quotation from a dated document with named members; it can be printed as such. "Mathematicians are furious" is a mood, and no artefact supports it. If you need the expert response in a sentence, find a named person and the words they used, or report that the response is still forming.