An AI research result is verified only with respect to a particular claim and a particular check. A theorem, a speed improvement and a successful demonstration need different evidence. Before repeating that an agent solved a problem, identify the statement, inspect the supporting artifact and record what the check leaves unresolved.
The matrix below gives research leads and technology writers a way to do that without assigning every result a single trust score. A machine-checked theorem can support a mathematical statement while leaving novelty or practical usefulness unsettled. A carefully measured experiment can answer a practical question without being a formal proof.
- 01Name the claim.A broad announcement may contain several assertions that need different checks.
- 02Keep evidence dimensions separate.Public artifacts, independent review and formal checking are different properties.
- 03Preserve the limits.Report the conditions, assumptions and broader conclusions that remain untested.
01 — Practical guideMatch the claim to the right check
Start with what would make the statement true or false. A formal proof establishes a proposition from stated assumptions. An experiment measures behavior under specified conditions. A demonstration shows an instance of behavior. These are useful distinctions, not rungs on a universal ladder.
Use each row to request an artifact and a matching check. A research project may need several rows: one for mathematical correctness, another for performance, and another for an end-to-end workflow.
| Claim | Artifact to request | Check and remaining limit |
|---|---|---|
| Mathematical theorem | Exact statement, definitions, assumptions and proof | Review the derivation and whether the statement expresses the advertised result. |
| Conventional argument | Complete reasoning, lemmas and dependencies | Qualified scrutiny of the argument; publication status alone does not close gaps. |
| Empirical improvement | Protocol, inputs, baseline, outcomes and analysis | Recompute the analysis and examine uncertainty; generalization remains a separate question. |
| Software behavior | Specification, code revision, tests and outputs | Check named requirements; passing selected cases does not cover every input. |
| Demonstrated capability | Full run, inputs and intervention record | Inspect the shown outcome; representativeness is still unknown. |
02 — Practical guideRead the statement behind a formal proof
A proof assistant checks a formally represented statement. The Lean 4 paper explains the system that represents theorems and proofs. That capability is not evidence that any particular AI-generated proof is sound: the actual artifact and its dependencies still matter.
Suppose an announcement says an algorithm works for every input, while the formal statement restricts its input to positive integers. A successful check supports the restricted proposition. It does not silently broaden the domain. This hypothetical example shows why statement review belongs beside proof checking.
Ask for definitions, assumptions and the environment used to check the result. Record any gap between the formal statement and the claim a reader is likely to understand. A missing proof and an explicitly declared assumption are different issues; neither should disappear behind a green build indicator.
The same distinction matters when agents generate and review many approaches. Our large-run orchestration analysis explains the role of independent acceptance. Here, the question is narrower: what did that acceptance actually establish?
03 — Practical guideRead measurements with their conditions attached
A performance claim needs a workload, a baseline and a measurement procedure. If an agent improves a program on one dataset, keep the dataset and conditions next to the result. A rerun using the same inputs can establish repeatability of that computation while leaving performance on new inputs unresolved.
The National Academies reproducibility report distinguishes recomputing results using the original materials from investigating the same question with new data. It also documents differences in terminology between disciplines. Define what your own record means instead of assuming that a label such as reproduced has one universal interpretation.
Do not turn artifact availability into a test result. A repository may make a check possible, but someone still has to execute the relevant procedure and retain its outcome. Likewise, a failure to retrieve an artifact is not proof that the underlying claim is false.
For benchmark-specific disclosures, use our vendor reproducibility audit . It covers information needed to inspect a score. Avoid duplicating its ledger when the question is whether a broader research claim has adequate evidence.
04 — Practical guideKeep checking method separate from reviewer status
Author-reported, peer-reviewed, publicly archived and independently checked describe different features. A result can have several of them at once. Record who performed which operation, with access to which materials, rather than collapsing these distinctions into verified or unverified.
The AlphaEvolve paper describes different evaluation mechanisms within one research system. Its optimization checks and expert correctness review should be reported as different activities. Those are the authors’ methods and outcomes; this reference does not reproduce them.
A peer-review decision is evidence about a review process. It does not replace an inspectable artifact or establish every secondary claim made in a press release. Conversely, an accessible counterexample can deserve examination before journal publication. Keep the conclusion proportionate to the actual checking mechanism.
When a result concerns practical speed, our guide to reading AI speedup claims helps identify the denominator. The same discipline applies here: specify the proposition that the evidence addresses.
05 — Practical guideCreate an evidence record another reader can inspect
Give each consequential claim its own record. Splitting a headline into separate entries prevents a checked mathematical statement from lending unearned certainty to an untested business forecast. Link records when they rely on the same artifact.
Use the blank claim record as a starting point. It contains field names, not validated research results. Include a source section or artifact revision so another reviewer can find the same evidence.
| Field group | Record | Why it matters |
|---|---|---|
| Claim | Exact statement, scope and conditions | Prevents a narrower result from being presented as a broader conclusion. |
| Artifact | Source URL, version, definitions or experimental inputs | Makes the underlying material locatable. |
| Check | Operation, checker, date and retained output | Separates reported checks from checks actually performed. |
| Independence | Relationship between claimant and checker | Describes provenance without inventing an aggregate trust rating. |
| Status | Not disclosed, inaccessible, not checked, passed or failed | Keeps missing information distinct from evidence of failure. |
| Limits | Assumptions, excluded conclusions and next check | Shows what a careful reader should still withhold judgment on. |
06 — Practical guideRevise the record when the evidence changes
A revised proof, a corrected dataset or an independent result can change the appropriate conclusion. Preserve the earlier record and explain the change. Do not silently replace an unsupported announcement with its corrected version and leave readers unable to see why the assessment moved.
Consider a hypothetical agent that announces a faster solver. The initial record might contain a demonstration and an author-supplied timing. A later record could add the workload, repeated runs and an independently rerun analysis. That progression improves the evidence for the measured speed claim; it still does not establish superiority on every workload.
Begin with the claim most likely to affect your next decision. Request its missing artifact or check, and state what remains unknown. For teams turning agent research into a working product, our AI transformation service can help connect acceptance criteria to the intended workflow.
Evidence and scope
- As-of date
- September 12, 2026. September 9 is the editorial allocation; this research was reviewed later.
- Method
- A definitional reference derived from primary research and standards. The matrix and blank record are Digital Applied editorial designs, not a scientific certification scheme.
- Limitations
- No mathematical proof or research experiment was reproduced. Historical papers provide the foundation; no present-day checker command or model ranking is asserted.
07 — Next stepAsk what was checked before deciding what to believe.
Ask what was checked before deciding what to believe.
Choose one consequential claim, identify its artifact and matching check, and record the remaining uncertainty. A useful evidence record makes the supported conclusion precise enough for another reader to challenge or reuse.