Give a coding agent the steps that expose the bug and the result that would count as a fix. “The form is broken” invites a guess. “Submit this safe sample from this starting page; an error appears and no confirmation is shown” gives the agent something it can observe before editing code, and something it can observe again afterwards to prove the edit worked.
The most useful bug report separates observation from diagnosis. You may suspect validation or a network problem, but keep that suspicion outside the reproduction steps. Otherwise the agent can produce a convincing fix for a cause nobody established, and the original failure survives untouched behind a green build. This guide sets out the seven-field packet, a worked example, why the acceptance behavior has to stay observable through the whole task, what to do when the bug will not reproduce, and how to close the report on evidence rather than on a confident summary.
- 01Preserve the starting state.The same clicks can behave differently with another account, record or browser state. The state is part of the reproduction, not background.
- 02Show expected and observed results.The gap between them defines the task better than a proposed implementation, and it is the thing the agent must close.
- 03Keep the diagnosis out of the steps.A suspected cause written as an instruction becomes the task. Record it separately so the agent can confirm or refute it.
- 04Verify the original complaint.A successful build is useful evidence, but it does not alone prove the reported defect is gone. Re-run the steps that exposed it.
01 — The ProblemA guess dressed as a report
A coding agent does what the report asks. If the report says “the validation on the contact form is broken, fix it”, the agent will find the validation code, change something plausible, run the tests and report success. Nothing in that sequence required the agent to see the failure, so nothing in it proves the failure is gone. The report converted a symptom into a diagnosis, and the agent treated the diagnosis as the task.
The alternative is a report the agent can execute before it edits anything. If it can reproduce the failure, it has a baseline; if it cannot, that is the first finding, and it is a useful one. Either way the agent’s work is anchored to an observation rather than to the reporter’s theory, and the same observation becomes the test that decides whether the work is finished.
Could someone who has never seen the code follow this report and watch the failure happen? If yes, the agent can too, and can watch it stop happening. If no, the report is asking the agent to trust a description, and the fix will be judged against the description.
02 — The PacketSend a compact reproduction packet
A reproduction is a repeatable way to observe the failure. Use the packet below, with safe sample data that preserves the relevant shape. These are recommended reporting fields, not a requirement to collect sensitive customer information; a sample record with the same structure as the failing one is almost always enough.
The right-hand column matters as much as the middle one. Each row’s anti-pattern is a thing reporters do when they are in a hurry, and each one turns the report back into a guess: a diagnosis disguised as a step, a preferred implementation disguised as an expected result, an invented percentage disguised as a frequency.
| Field | Useful content | What to avoid |
|---|---|---|
| Environment | Page, app revision and relevant browser or device | A vague claim that it fails everywhere |
| Starting state | Required sign-in state and safe sample record | Live credentials or unnecessary personal data |
| Steps | The shortest ordered actions that still fail | A diagnosis disguised as an instruction to change code |
| Expected result | Observable behavior grounded in a requirement | A preferred implementation without a user outcome |
| Observed result | Exact error, screenshot or relevant log reference | A screenshot with no action or time context |
| Frequency | Observed attempts and conditions | An invented failure percentage |
| Verification rule | Repeat the original steps and check adjacent behavior | Only checking whether a file was edited |
03 — The ExampleDescribe one failure without guessing its cause
Consider an illustrative contact form. With a valid sample address and a message, selecting Submit displays a spinner that never clears. A useful report states the page, starting state, inputs, action and visible result. It also says whether a confirmation message was expected under the current requirement, because “the spinner should clear” and “a confirmation should appear” are different acceptance rules and the agent needs to know which one it is working to.
The spinner does not prove the request failed. A diagnostic check might reveal that the submission was saved while the response handler failed, in which case the user-visible bug is a missing confirmation and the fix is nowhere near the validation code the reporter suspected. Repeating live submissions could create duplicates, so reproduce with controlled test data and inspect the resulting state.
Attach the smallest relevant evidence. A timestamp helps correlate a log entry; a screenshot shows what was visible. Neither substitutes for the steps. For unknown completion, the tool-error reference provides a useful decision boundary between “failed”, “succeeded” and “unknown”, and the third state is the one this example is in until someone looks at the destination.
What the reporter saw
Contact page, signed out, sample address and message, Submit pressed, spinner never clears, no confirmation. Expected: a confirmation message within a few seconds. Seen on two attempts today.
What the reporter suspects
“Probably the new validation rule.” Written in its own field so the agent can test it, not in the steps where it would become the task.
What the agent discovered
The record was saved on both attempts; the response handler throws on an undefined field. The validation rule was not involved. The fix is in the handler and the test is the original steps.
04 — The AnchorKeep the acceptance behavior observable
Anthropic’s long-running harness report describes explicit feature lists and browser checks used to address premature completion. The lesson we apply here is to keep the requested behavior observable throughout a coding task: the agent should be able to run the reproduction at the start, during, and at the end, and “done” should mean the last run behaved differently from the first.
Its agent-evaluation guide emphasizes well-specified coding tasks, stable test environments and tests of the generated code. A customer report becomes more useful when it provides the inputs for such a task. The packet is, in effect, a one-case evaluation: a fixed starting state, a fixed action, and a grader that checks the observed result against the expected one.
Observable also means observable by someone other than the agent. A fix verified only by the agent’s own description of what it saw is a narrated success. Ask for the command, the screenshot or the state query that a reviewer can run, and keep it in the report so the verification outlives the session that produced it.
05 — The Vanishing BugPreserve uncertainty when the bug disappears
If the agent cannot reproduce the issue, keep the report open as unreproduced. Record what was tried and under which conditions. Ask for the missing detail that could distinguish the failing case: account state, sample shape, timing or a particular revision. An unreproduced bug is a report with a missing field, not a closed one.
Do not silently broaden the claim from one report to a universal defect. Conversely, one successful attempt does not refute an intermittent complaint. A useful next step is focused instrumentation or another controlled attempt with the relevant conditions preserved, and the frequency field of the packet is where the attempts and their conditions are recorded so the pattern can emerge.
Protect the report from speculative edits. A refactor that makes the code look cleaner may be worthwhile for another reason, but it is not evidence that the original defect was fixed. The reviewer evidence guide explains why a second approving opinion does not settle that question either.
06 — The FailuresFour ways a fix is not a fix
Each pattern below passes an ordinary review and leaves the user’s failure in place. They are worth naming because the agent’s summary of each one reads as success.
The adjacent fix: the agent changes the code the report blamed, the build is green, and the spinner still spins because the cause was elsewhere. The suppressed symptom: the spinner is given a timeout and a generic message, so the visible failure is gone and the saved record and the thrown error are untouched. The accommodating test: a failing assertion is adjusted so the suite passes, and the adjustment quietly redefines the expected result. The clean-code detour: a tidy refactor of the surrounding module, reported as “fixed and improved”, with the reproduction never re-run.
One check catches each of the four: re-run the original steps from the original starting state and compare the observed result to the expected one in the packet. If the report was written so the agent could do that, the report also makes it possible for a reviewer to do it in a minute.
“Show me the reproduction failing before your change and passing after it.” A fix that cannot be shown this way may still be a good change. It is not yet evidence that the reported bug is gone.
07 — The CloseClose with evidence tied to the original steps
At closure, record the revised code’s result on the original reproduction and any adjacent behavior the change could affect. If the agent needed to change the acceptance rule because the original expectation was wrong, document that decision and its owner. Do not alter a test merely to make a failing result disappear; a changed rule is a product decision and needs a product owner’s name next to it.
Separate local verification from publication. A tested change still needs the project’s release process, and a fix that works on the agent’s machine has not yet worked for the user who reported it. When work moves to another agent or reviewer, use the handoff ownership guide to preserve the remaining responsibility, and our guide to showing changes before publishing for the release step itself. Teams that want the packet built into their intake so every report arrives reproducible can work with our AI transformation practice.
08 — DecisionA bug is closed when the reproduction stops reproducing
Make the reported failure the acceptance test.
Send the agent a reproducible observation, keep the diagnosis provisional and close the report only against the original behavior or an explicitly revised requirement. Everything else the agent does along the way is work; only that last comparison is evidence.
For implementation support, explore our AI transformation services.