What is hallucinated success?
Hallucinated success is when an AI agent reports an action as done, such as "order created" or "refund issued", although the write failed, only partly happened or never ran. It is also called a false "done". It is dangerous because nothing looks wrong: the user gets a confident confirmation and the logs show a tool call.
What it looks like
- Failed write, success message. The API returned an error; the agent said it worked.
- Partial write, full success message. The order header was saved, the line items were rejected. See partial writes.
- No write at all. The agent skipped the tool call and described the result anyway.
- Wrong write, right-sounding message. The confirmation quotes the value the agent intended, not the one saved. See wrong data.
Why it happens
The model never sees the database. It sees tool output as text. A wrapper that swallows an error, a 200 response with an error in the body, a timeout with no response, or an error many steps back in a long context can all make "done" the most plausible next message.
How to catch it
Compare what the agent claimed with what the system holds. After each key task, read the record back: does it exist, with the right values, exactly once? Then force the conditions that cause it, with injected server errors and timeouts. See fault injection for AI agents.
An agent is allowed to fail. It isn't allowed to fail and say it succeeded.
On the blog
- AI agent hallucinated success
- AI agent partial failures and rollback testing
- How to test AI agents that write to your CRM or ERP
Related terms
Partial writes · Duplicate writes · Fault injection for AI agents · All terms
FAQ
A trace shows that the agent called the tool and got a response. It doesn't prove the record exists with the right values. You have to read the record back.
Not reliably. The judge reads the same transcript and the same ambiguous tool output, so it can make the same mistake. The check has to come from the system of record.
It is a kind of it. Instead of a made-up fact in an answer, the agent makes up the outcome of an action it took.
Try it on your own agent
Describe a task in plain words. Agentiqa does it step by step and tells you if it worked. Every failure comes with step-by-step reproduction.
20 runs a month, no credit card.
Your agent writes into ERP, CRM or billing for customers? See Enterprise.