In short: to test an AI agent's tool calls, run realistic requests end to end and check four things for each call: that the agent picked the right tool, passed the right arguments, called tools in a valid order, and handled the tool's errors honestly. Then read back the system the tool wrote to, because a well-formed call can still change the wrong record. Mock tools are fine for fast checks on arguments; the release decision needs real calls against a test instance of the target system.
Most agent bugs that reach customers are not wrong answers. They are wrong actions: a call to the right API with a wrong field, a second call nobody expected, or an error the agent swallowed before telling the user "done". This post covers how to test that layer.
What does testing a tool call actually check?
A tool call is the point where the agent stops talking and starts changing things. Each call has four properties you can test:
| Property | The question | Typical failure |
|---|---|---|
| Selection | Did the agent pick the right tool? | Uses "update" where it should "create", or searches instead of writing |
| Arguments | Are the values right? | Wrong supplier ID, wrong unit, wrong date format, a field left empty |
| Sequence | Is the order valid? | Writes before it checks, approves before the amount is final |
| Error handling | What happens when the tool fails? | Retries into a duplicate, or reports success anyway |
Most teams test selection and stop there. The expensive failures sit in arguments and error handling.
Why isn't checking the agent's reply enough?
The reply is the agent's description of what it did, not a record of it. An agent can say "I created the order for 40 units" while the call sent 4, or while the call timed out. Tests that grade the reply miss both cases. The background is in AI agent hallucinated success.
Tool-call tests look at two other sources instead: the trace of calls the agent made, and the state of the target system after the run.
How do you test which tool the agent picks?
Write requests that should map to one tool, and a few that sit near the border between two tools. "Change the delivery date on order 1182" should call an update, not create a new order. "Order the same as last month" should look up the last order before it creates anything.
Check the trace for the expected tool and for tools that should not appear at all. An agent that calls a delete or send tool during a read-only request has a selection problem, even if the final result looks fine. Whether the agent can reach such a tool at all is a permissions question; see AI agent permission testing.
How do you test tool-call arguments?
Write the expected arguments next to each request, field by field, and compare them with the trace. Pay attention to the fields that cause real damage when they are wrong:
- IDs: the customer, supplier, matter or account the write belongs to;
- amounts and units: quantities, currencies, pieces vs boxes;
- dates: time zones, date formats, deadlines computed from other dates;
- free text that gets stored: descriptions and notes that later reach a customer.
Then check the target system. Some tools change or default values on their own, so a correct argument can still produce a wrong record. The record is what counts. The field-level approach is in AI agent wrote wrong data to your CRM or ERP.
How do you test the order of tool calls?
Not every flow has one correct order, so don't assert the exact sequence unless it matters. Assert the rules instead:
- lookups that the write depends on happen before the write;
- a check or approval step happens before the action it guards;
- nothing is written after the agent decided to stop;
- the same write does not appear twice.
Rules like these survive prompt and model changes that reorder harmless steps, so the tests break less often for the wrong reasons.
How do you test tool errors and timeouts?
Make the tools fail on purpose and watch what the agent does. Useful failures to inject:
- the API returns an error on the first call;
- the call times out after the write went through;
- the tool returns an empty or partial result;
- a later step fails after an earlier one already wrote data.
For each one, decide in advance what correct behavior looks like: retry safely, stop and report the failure, or ask a human. Two outcomes are always failures: a duplicate record after a retry, and a success message after a failed write. See duplicate writes and idempotency testing.
Should you mock tools or call the real system?
Both, for different jobs.
| Mocked tools | Real test instance | |
|---|---|---|
| Speed | Fast | Slower |
| Good for | Selection, arguments, sequence rules, injected errors | Defaults, validation, side effects, the written record |
| Misses | Everything the real system does to the data | Little, if the instance is configured like a customer's |
| Use it for | Quick checks while you change prompts | The release decision |
A mock only knows what you told it. Real systems apply defaults, round values, trigger workflows and reject inputs in ways a mock doesn't. That's why the check before a release runs against a test instance configured like a real customer's.
How do tool-call tests fit into a release gate?
Tool-call checks are one layer of a release gate for agents with write access. The gate runs the agent's critical flows end to end, checks the calls and the written data, and stops the release on a mismatch. The full method is in how to test AI agents that write to your CRM or ERP, and the definition of the gate itself is in what is a quality gate for AI agents.
Rerun the tool-call tests whenever a prompt, a model, a tool definition or the API behind a tool changes. Model updates often change how arguments are filled even when the reply reads the same; see model update broke your AI agent. A ready list of cases to start from is in AI agent test cases: 12 examples.
How does Agentiqa help?
Agentiqa builds high-confidence release gates for vertical AI agents in high-liability domains such as procurement, legal, finance and supply chain. It runs the agent's flows end to end, checks the actions the agent took and the data it wrote, and stops the release on a mismatch, with the trace and the data diff as evidence. First release gates live within 7 minutes. For teams starting with web apps or simple agentic workflows, the free Community Edition is a way to try this kind of testing yourself.
FAQ
What is AI agent tool call testing?
It is testing the actions an agent takes through its tools: which tool it picks, which arguments it passes, in what order it calls tools and how it handles tool errors. It checks the written result in the target system, not only the agent's reply.
How do I test an AI agent that calls APIs?
Run realistic requests, record the API calls the agent makes, compare tools and arguments with what you expected, and read back the system the API changed. Add cases where the API fails or times out.
Can I test tool calls without a real system?
Partly. Mocked tools are good for checking selection, arguments and error handling quickly. They can't show what the real system does with the data, so the release check needs a real test instance.
How do I make tool-call tests stable when the agent is non-deterministic?
Assert outcomes and rules, not exact traces: the right record exists, required lookups came before the write, nothing was written twice. Run critical flows more than once and treat inconsistent results as a finding.
Can I try this for free?
Yes. The Community Edition is free, with 20 runs a month that reset monthly. Pro is $39/month.
Next step
Pick the one tool your agent uses to write the most expensive data. Write down three requests that use it, the arguments you expect for each, and what should happen when it fails. That is your first tool-call test. If you want help turning it into release gates, we map it with you in 20 minutes.