AI agent testing glossary
Short, plain definitions of the terms we use when we write about testing AI agents that write data into business systems. Each term links to the longer posts on the blog.
Terms
| Term | Definition |
|---|---|
| AI agent release gate | A set of pass/fail checks a new version of an AI agent has to pass before it goes to production. Read the definition |
| Quality gate for AI agents | The same idea seen from the release process: the agent's critical actions run before a new version reaches production. Read the guide |
| AI agent evals vs testing | Agent evals score an AI agent's outputs on a dataset, with code checks or an LLM judge, usually with mocked tools. More |
| AI agent regression testing | AI agent regression testing means re-running an agent's key tasks whenever its prompt, model, tools or integrations change, and checking the end state in the systems it writes to. More |
| AI Quality System | An AI Quality System is the set of checks, people and evidence a team uses to decide whether a new version of an AI agent is safe to ship. More |
| Duplicate writes | Duplicate writes happen when an AI agent's write succeeds but the response is lost, and the agent, its SDK or the model retries, so the same action is saved twice: a second purchase order, a second refund, a second journal entry. More |
| Fault injection for AI agents | Fault injection for AI agents means making the agent's tools return server errors, timeouts and malformed responses on purpose during a test, then checking what the agent did and what it left in the connected system. More |
| Hallucinated success | Hallucinated success is when an AI agent reports an action as done, such as "order created" or "refund issued", although the write failed, only partly happened or never ran. More |
| Partial writes | A partial write is when an AI agent completes some steps of a multi-step task and then stops, leaving a half-finished change in a business system: an order without its lines, a refund without its credit note, a contact moved without its deals. More |
| Prompt injection testing | Prompt injection testing for AI agents means running cases where instructions are hidden in content the agent reads, such as an email, a supplier document or a web page, or where a user pushes it past a rule, and checking that it refuses, escalates or fails safely. More |
| Shadow mode testing | Shadow mode testing means running an AI agent on real requests while its writes are held back, then comparing what it would have done with what people actually did. More |
| Vertical AI agent | A vertical AI agent is an AI agent built for one business process, such as procurement, legal, finance or supply chain, that takes actions in the systems of that domain: it creates purchase orders, files documents, posts entries or places replenishment orders. More |
| Wrong data | Wrong data is when an AI agent writes to the right record and the write succeeds, but the values are off: a truncated ID, pieces instead of boxes, an amount in the tax field, two swapped dates. More |
The release gate and quality gate entries link straight to their existing definition posts, so there is one page per definition.
FAQ
Engineers and product teams who ship AI agents that create or change records in ERP, CRM, billing and other business systems.
Because each one needs a different test. Wrong data needs a field-by-field read-back, duplicate writes need injected timeouts, and hallucinated success needs a comparison of the claim with the record.
Agentiqa's release gates run an agent's key tasks before each release and check the records in the connected system. You can try Community Edition free, with 20 runs a month.
Try it on your own agent
Describe a task in plain words. Agentiqa does it step by step and tells you if it worked. Every failure comes with step-by-step reproduction.
20 runs a month, no credit card.
Your agent writes into ERP, CRM or billing for customers? See Enterprise.