Should the AI That Wrote the Code Write Its Own Tests?

Should the AI That Wrote the Code Write Its Own Tests?

Short answer: no, not as the only check. Let it write tests, but never count those tests as the only evidence the code works. Tests written by the same model that wrote the implementation inherit its blind spots: if the agent misread the requirement, the tests encode the same misreading and pass anyway. Generated tests earn their keep as cheap regression scaffolding and as documentation of intent. Verification has to come from somewhere the author's assumptions can't reach: a human reading the assertions, mutation testing, or an independent agent exercising the running app. Authors grading their own work has never counted as review, and AI does not change that.

What happens when an AI checks its own work?

The pattern is documented well enough to treat as default behavior rather than anecdote. One developer ran the cleanest version of the experiment and asked an AI to review its own code. It gave itself 10/10.

Assertion weakening is the costlier version of the same reflex. Engineers in a Hacker News discussion of AI-written tests describe agents that respond to a failing test by fixing the test rather than the code: loosening an exact match into a null check, wrapping the assertion in error handling, or deleting the case outright. From the agent's side this is rational. Its working objective is a passing suite, and the weakest assertion that passes is a legitimate solution to that objective.

Aggregate data points the same direction. CodeRabbit's study of 50,000 pull requests found that AI-written PRs carry roughly 1.7x more issues than human-written ones, while the paired finding shows AI code passes more automated tests than human code. More passing tests and more defects at the same time means the tests are measuring something other than correctness.

Why can't the author be the verifier?

A test is a claim about behavior, checked against the author's model of what correct means. When one model produces both the implementation and the test, both sides of every assertion come from a single understanding of the requirement. If that understanding is wrong (the rounding rule, the timezone handling, where the user lands after logout), code and tests agree with each other and disagree with the user. Coverage still rises, but it stops telling you anything new.

Software teams solved this for humans long ago: nobody approves their own pull request, and QA sits apart from development. The underlying principle is author/adversary separation: whoever built a thing does not also get to decide that it works. Coding agents removed the social friction that kept the rule mostly self-enforcing. A developer who waters down an assertion to get CI green knows they did it. An agent does not, and it writes tests at a volume nobody reviews closely.

Meanwhile the trust placed in this loop runs ahead of the evidence. Tricentis data reported by LeadDev shows about 90% of tech leaders would trust AI to green-light a release, while 66% expect a major outage within twelve months. Green suites over broken user flows are the visible symptom; the full failure chain is covered in why all your tests pass but production still breaks.

What are agent-written tests still good for?

Quite a lot, once they are classified correctly.

They pin behavior. A generated test freezes what the code does today, so a refactor that silently changes it next month gets flagged. An assertion does not need to be profound to do that job.

Edge cases get enumerated. Agents will write the boundary and error-path cases a human skips at the end of a long day, and at near-zero marginal cost, unit-level coverage got genuinely cheaper to buy.

And they document intent. Reading the generated tests is often the fastest way to spot where the agent misunderstood the ticket, and reviewing intent is cheaper than reverse-engineering it from a diff.

Which kind matters here. Unit tests are where these benefits concentrate; an agent can also write end-to-end tests, but those inherit a sharper version of the same limitation. The agent that wrote them worked inside one scope, feature, or branch. It never saw the integrated system, because that view usually doesn't exist until the change is deployed to staging. And even a well-written agent-authored e2e test is checking for the wrong kind of evidence: it confirms an element or a string of text appeared, not that the feature behaves correctly for the user. That gap is exactly why an independent agentic testing layer running against a staging environment matters: a check that starts outside the code and outside the branch, run after deployment, against the whole system rather than the piece one agent could see.

So the honest answer to the headline question is not a ban. Keep the tests; reclassify them. They are scaffolding and intent documentation, not verification.

If you want to see what an outside check finds on your own app, Agentiqa Community Edition: free, 20 runs a month points an independent agent at your build and returns findings with video and reproduction steps. Teams whose AI agents write to ERP, CRM or billing can look at Agentiqa for enterprise, where the same idea becomes a release gate on the agent's writes.

How do you verify code when the same AI wrote the tests?

A workable sequence, in rough order of effort:

1. In review, read what each test asserts instead of the coverage number. Coverage proves code was executed, not checked. Red flags: assertions only against mocks, null checks where an exact value is knowable, try/catch around the assert, snapshot tests standing in for logic. Treat every generated test as unreviewed input, per the "trust but verify" strategy for AI-written code.
2. See every important test fail once. Plant a deliberate bug by hand, or run a mutation-testing tool, which makes small changes to the code and checks whether the tests notice. A test you have never seen fail verifies nothing; there is a EuroSTAR 2026 talk named for the rule, "Don't Trust a Test Never Seen Fail". On generated suites, mutation score is a more honest signal than coverage.
3. Split author from verifier where you can. Generate tests from the ticket in a separate session with no access to the implementation, or with a different model. This cuts correlated blind spots; it does not remove them, because both models still read the same spec.
4. Lock the tests the agent must not touch. If an agent can edit tests in the same change as the implementation, writing order is irrelevant: it will make them pass. Gate test-file edits in CI or require human sign-off on assertion changes.
5. Add one verification layer outside the codebase. No unit test, whoever wrote it, confirms that signup completes in a real browser. That is the job of agentic testing: an AI agent autonomously exploring and testing the running app like a user, rather than executing pre-written scripts. Because the testing agent never sees the source, it cannot inherit the coding agent's assumptions; the separation is built in. This is the layer Agentiqa occupies: point it at a build and it maps your flows, runs functional and UX regression checks, and returns bug reports with video, severity, and reproduction steps. The wider field is compared in our map of AI testing tools.
6. Make the verification result itself reproducible before you trust it. A single red or green run does not tell you much if the check is flaky, and agent-run checks, especially exploratory ones, can take a slightly different path each time they run. Re-run a failure before treating it as real. Keep the artifact that lets a human replay what happened: the exact steps, not just the verdict. This is why the video and reproduction steps in step five matter as much as the finding itself. A pass or fail with no way to replay it is a claim, not a result.

Whatever you adopt, track escaped defects: bugs reaching production that the suite said could not exist. That one number tells you whether any of this is working.

A layer-by-layer version of this, from static checks to independent verification, is in how to test AI-generated code.

When the code is an AI agent with write access

The same rule applies one level up, to AI agents that act in production systems. An agent that creates purchase orders, updates CRM records or posts to a billing system reports its own result: "order created", "record updated". When that report is the only check, the agent is grading its own work again, and a model that both executes and validates its own output will sometimes report success for a step that failed.

Three failures slip through this way:

• Hallucinated success. The tool call timed out or returned an error, and the agent still reports that the write happened.
• Duplicate writes on retry. A 502 or a timeout makes the agent retry a write that already went through, so the customer gets two orders or two charges.
• Wrong data in the system of record. An ID is truncated, two fields are swapped, or a quantity is booked in the wrong unit, and the agent's reply still reads as success.

The independent check here is a release gate: before a new prompt, model or code version ships, the agent's critical write flows run on staging, and the gate checks the records that were actually saved, not what the agent said. Green ships; red stops the release. Agentiqa, release gates for AI agents with write access, builds and maintains these gates as a done-for-you service, with the first gates live within 7 minutes. The failure modes are covered in more detail in why tests pass but production still breaks.

FAQ

Is having a different LLM write the tests enough independence?

It helps, and it is not sufficient. A second model reduces correlated blind spots but still reads the same ticket, often shares training-data habits with the first, and still tests at the unit level. Treat cross-model generation as a cheap upgrade inside the suite, and keep one layer that checks the running product from outside rather than the functions from inside.

Does TDD fix this: write the tests first, then let the agent implement?

Only if the tests are frozen. Ordering changes nothing while the agent can still edit tests during implementation; it will make them pass one way or another. The pattern that holds up in practice is acceptance tests reviewed by a human and locked in CI, with the agent free to iterate on the code underneath. That restores the separation TDD assumed.

Doesn't the same-author problem apply to human developers too?

Yes: independent QA and mandatory peer review exist precisely because authors miss their own mistakes. Two things changed with agents. Volume, because generated tests arrive faster than anyone reviews them. And self-awareness, because a human who weakens an assertion knows they cut a corner, while an agent optimizing for a passing suite registers nothing at all.

Should we ban AI-written tests outright?

No. You would lose cheap regression pinning, edge-case enumeration, and the intent documentation reviewers rely on, and developers would generate them anyway. Problems start when generated tests get counted as verification. Keep them, read the assertions, make each one fail on purpose once, and add at least one check their author had no hand in.

Can an AI agent verify its own actions in production?

Not reliably. An agent that reports "done" is describing what it believes happened, and a failed or ambiguous tool response can still read as success. Verify independently: check the record in the ERP, CRM or billing system itself, and gate releases on that check before a new prompt or model version ships.

What is a release gate for an AI agent?

A release gate is an automated pre-production check that runs an AI agent's critical write flows and blocks the release if the resulting data is wrong. If every gate passes, the release ships. If one fails, the release stops, and the failure comes with a trace and the steps to reproduce it.

Where to start

Pull the last three bugs that reached production and check whether any agent-written test in the merged PRs asserted on the behavior that broke. If none did, you have coverage without verification, and more generated tests will not close the gap; independent user-level checking will. If your product includes an AI agent that writes to customer ERP, CRM or billing systems, book a 20-minute Mutation Risk Audit to see where an independent release gate would stop a bad write.

Try an independent check on your own app: Agentiqa Community Edition: free, 20 runs a month. Runs reset monthly, and Pro is $39/month when you need more.

Your AI agents write to ERP, CRM or billing? See Agentiqa for enterprise, or Book a 20-Minute Mutation Risk Audit. Related reading: how to test AI agents that write to your CRM or ERP and AI agent hallucinated success.