Share
From Shadow Mode to Write Access: Checks Before You Graduate an AI Agent
November 16, 2026Read time: 8 min

From Shadow Mode to Write Access: Checks Before You Graduate an AI Agent

A clean shadow log is not proof a write will be correct. Stages, evidence and limits for moving an AI agent from shadow mode to write access.

In short: Shadow mode shows what an agent would have done, without writing. That is useful, and it is not proof that a write will be correct. Graduate the agent in stages — read, shadow, limited write, then a wider write — and require evidence at each stage before you loosen the limit.

The earlier post, shadow mode testing for AI agents, covers what shadow mode catches and what it misses. This guide is the next decision: when, and with what evidence, to give the agent write access, and how to widen that access without treating a clean shadow log as a release.

What can shadow mode prove, and what can't it?

In shadow mode the agent sees the same request and the same systems, and it proposes an action, but the write is not applied. You can compare the proposal with what a person did, or with a pass rule you already wrote.

It can catch a wrong target, a wrong amount, a skipped check, or a proposal that a person would have refused. It cannot catch faults that only appear when the write is real: a retry that creates a second row, a confirmation the live system shows only after submit, a validation the interface or the API applies only on write, or a downstream record that changes as a side effect. Those paths were not taken.

So a clean shadow period is evidence about proposals. It is not evidence about writes. How to test AI agents before they get ERP write access stays relevant at every later stage, because the write still has to be tested for real, on a test copy, before it is allowed on customer data.

What are the stages, and what evidence does each one need?

Move one stage at a time. Stay on a stage until the evidence in the right-hand column exists. Do not skip a stage because a public benchmark looks strong: EnterpriseOps-Gym (ServiceNow Research, March 2026) ran 1,150 stateful enterprise tasks across 14 models, and the best result was 37.4% (Claude Opus 4.5). That figure is about those tasks, not about yours.

Four-stage flow for graduating an AI agent from read-only and shadow mode to limited write and then wider write, with evidence required before leaving each stage.

StageWhat the agent may doEvidence before you leave this stage
1. Read onlyLook up records; no proposals that a person might mistake for a write, and no write credentialsThe agent cannot call a write tool or submit a write screen; a deliberate "try to write" case is refused and nothing changes
2. ShadowPropose writes; a person or a separate process applies them, or they are discardedA fixed set of tasks has been compared against a pass rule or a person's decision, including cases the agent should refuse; misses are written down
3. Limited writeWrite only in a test copy, or in production on a narrow, named set of records, with a person approving each writeEvery allowed write task has a record-level pass rule; those cases pass on a test copy, including retry, timeout, lookalike and refusal cases; human approval is itself tested
4. Wider writeWrite without per-action approval on the tasks that already passed stage 3, still under least privilegeThe same cases still pass as release gates on a test copy before each release; a rollback path is tested for a partial write (partial failure and rollback)

"Limited" needs a written limit: named record types, a maximum amount, a single workspace, a time window, or a person in the loop. A limit that exists only in a prompt is not a limit you have tested.

What should you require before the first real write?

  • Pass rules in the system, not in the log. "One update to supplier S-1042, payment terms 30 days, S-1043 unchanged" is a rule you can read back. "The proposal looked right in shadow mode" is not.
  • Refusal cases, not only happy paths. EnterpriseOps-Gym also reports that the best clean refusal rate on infeasible tasks was 53.9%. Whatever the rate on your tasks, include cases the agent must not write: a missing record, a held account, a request outside its role.
  • Retry and timeout cases on the test copy, before any production write. Shadow mode did not take this path.
  • Least privilege for the write credentials, checked with a case that tries to go outside the limit. See AI agent permission testing.
  • A named owner who can revoke the write, and a way to turn the agent back to shadow or read-only without a model change.

Worked example (illustrative): a finance agent has spent a shadow period proposing payment-term updates. The first limited write is allowed only in the test company code, only on suppliers in a named test group, and only with a person approving each change. Graduation to a wider write waits until the test-copy cases — including a retry and a lookalike supplier — pass, and until a refused update (a supplier on hold) writes nothing.

What should make you go back a stage?

Move back to shadow, or to read-only, if any of the following is true:

Four reasons to move an AI agent back from write access to shadow or read-only, including a wrong or duplicate write, an untested change, weak approval, or a task with no pass rule.

  • A write in the limited set was wrong, duplicated, or outside the task, including after a retry.
  • The target system, the tool list or the agent's instructions changed, and the cases have not been rerun.
  • The person in the loop started approving without reading, or the approval step was removed without a new test.
  • A task was added that has no record-level pass rule.

Going back is a control, not a failure of the plan. The stages exist so you can tighten the limit without inventing a new process under pressure.

How Agentiqa fits

Agentiqa is an AI Quality System for production AI agents. With Agentiqa, you describe each task in plain words. When you rerun the failed tasks, AI Quality Manager reports a step as a regression only if it fails again. Unconfirmed failures are listed separately.

First release gates are live within 7 minutes; for enterprise teams, an Agentiqa engineer sets up the gates on the riskiest tasks over a 3-day rollout.

FAQ

When should you give an AI agent write access?

After you have record-level pass rules for the tasks it will write, those cases pass on a test copy (including refusal, retry and lookalike cases), and the first live writes are limited and approved by a person. A clean shadow log is not that evidence.

Is a clean shadow-mode period enough to allow writes?

No. Shadow mode does not execute the write, so it cannot show duplicate rows, submit-time validation or side effects that appear only after a save. Use it to grade proposals, then test the write for real on a test copy.

How narrow should the first write access be?

Narrow enough that a bad write is contained: a test copy, or a named set of records, with a person approving each action. Widen only the tasks that already have passing cases and a release gate.

What if the agent scores well on a public enterprise benchmark?

Use the benchmark as context, not as graduation evidence. In EnterpriseOps-Gym, the best of 14 models reached 37.4% on 1,150 stateful enterprise tasks. Your pass rules, on your records, are the evidence that matters.

Should write access start in production if shadow mode already ran there?

No. The first executed writes belong on a test copy. If you later allow a limited production write, keep the same pass rules and the person-in-the-loop step until those production writes have a clean, reviewed history of their own.

Sources

  • ServiceNow Research, "EnterpriseOps-Gym: Environments and Evaluations for Stateful Agentic Planning and Tool Use in Enterprise Settings", March 2026. arXiv:2603.13594. 1,150 tasks, 14 models; best result 37.4% (Claude Opus 4.5); best clean refusal of infeasible tasks 53.9%.

Next step

Write the stage your agent is in today, the limit you will give it next, and the record-level pass rules that have to pass before you move. If you build or run vertical agents and want to turn those rules into release gates, we can go through the riskiest tasks with you in 20 minutes.

Book a 20-Minute Vertical Agents Safety Review

Related posts