An AI Agent in Production Is an On-Call System, Not a Feature
The page arrives at 3:07 a.m. and reports that an ingestion agent quarantined 63,402 rows overnight because it judged a supplier's renamed columns to be an attack on the schema. The engineer holding the phone did not write the agent. What she can read of its reasoning is a chat transcript that ends with the agent apologizing, confidently, and asking whether it should resume. That transcript is the entire incident record, and it reviews like a story told twice.
The claim, written to be challenged: once an agent takes actions on real systems while nobody watches, it stops being a feature and becomes an on-call system in every sense that matters. On-call systems survive the night on boring properties, and agents inherit the same list. We build agentic systems for data work, so the question we now put to every deployment is the one infrastructure teams have asked for decades: when this behaves oddly at 3 a.m., who reads what?
Odd behavior is an operational event, and the chat is not its report
An operational event has standard fields: severity, blast radius, reproduction, owner. A chat bug has a different shape: regenerate and move on. Treating the first like the second turns a wrong quarantine into a worse morning, because the remediation for a botched summary is another regeneration, while the remediation for a wrong destructive action is a rollback you can no longer plan without evidence.
So log the event as the event it is. Which agent version ran, which tools it called with which arguments, what changed in the world, and who owns the investigation now. Alongside that, one dead end we would like to spare you: asking the agent to draft the incident summary itself. It will oblige. The draft is plausible, organized, and occasionally wrong in the quiet direction of the agent's own innocence, and the reader cannot tell which drafts are which. Plausible is not evidence.
A trace must replay, not narrate
The record that survives an investigation is a replayable trace, and its test is narrow: can a second engineer re-run the run, step by step, outside the agent, and watch the same decisions happen again? A transcript fails that test three ways. It narrates instead of measuring, it drops the arguments it found boring, and it does not say which version of the system produced it.
The shape is unromantic, closer to an access log than to a diary:
{
"run_id": "ing-2026-03-17-4471",
"agent_version": "ingest-triage@v14",
"prompt_version": "git:2f9c1ab",
"model": "claude-sonnet",
"steps": [
{
"tool": "read_sample",
"args": { "table": "supplier_shipments", "rows": 50 },
"output_ref": "traces/ing-2026-03-17-4471/step-3.json",
"started_at": "2026-03-17T09:14:22Z"
},
{
"decision": "quarantine_table",
"rationale_ref": "traces/ing-2026-03-17-4471/step-5.json",
"reversible": false
}
]
}
Large outputs live behind references so the trace itself stays readable, and reversible is written at decision time, not reconstructed afterward. The two fields teams forget are the agent and prompt versions. Without them, a trace replays a system that no longer exists, and the replay disagrees with the incident in ways nobody can attribute. Versions ship with the service, and these are part of the service.
A trace that replays is evidence. A transcript that narrates is testimony.
Escalation names a human, and the trigger is the cost of being wrong
Every agent with write access carries an escalation policy that answers three questions with names: what triggers the page, who holds the phone, and what the agent is forbidden from touching while the page is open. The trigger that fails first is confidence. A model is least calibrated on exactly the novel situations where escalation matters, so confidence-based paging produces silence on the dangerous nights and noise on the quiet ones, sometimes within the same week. The cost of a wrong action is the honest trigger: regenerating a summary is reversible, dropping a table is not, and the paging budget follows that line.
The working rule we land on: retry a failing step at most once, then page, with the trace reference attached so the human starts from evidence instead of from the chat. Every escalation lands in the same queue your service incidents use, with the same acknowledgment expectations. The review that follows is an incident review, not a thread drifting toward Friday.
The kill switch is scoped, or nobody pulls it
A total stop button that nobody has rehearsed, on a system whose downtime you never budgeted, will not get pulled. The switch that works is scoped, because most agent incidents do not need the agent stopped. They need its write access gone while its reading continues to help the human who just got paged:
kill_switches:
- target: ingest-triage
scope: tool
tool: write_table
effect: deny_writes_keep_reads
may_pull: [oncall-primary, oncall-secondary]
restored_by: agent-owner
- target: ingest-triage
scope: agent
effect: drain_decisions_to_human_queue
may_pull: [oncall-primary]
restored_by: agent-owner
Rehearse it during a quiet week. Watch the queue fill, watch the product degrade the way the runbook promises, and time the restore, because a switch whose restore path is a mystery stays pulled long past its need.
A kill switch never thrown in rehearsal is a rumor.
What a runbook cannot fix
Limits first. A trace replays only what was logged, and agents fail in ways nobody thought to log, so the first hour of any novel incident is still a person reading. The runbook itself rots: review it with the postmortems attached, and let every incident retire one unclear entry, or the entries go soft in exactly the sections nobody has exercised. And the deepest limit: runbooks make an agent survivable, not correct. Correctness is the agent's problem; survivability is yours, and the two get fixed by different commits.
Our own regret belongs here. We once shipped a document-triage agent that kept only its transcripts, the transcript storage rotated them out on a schedule nobody had connected to the agent, and a mis-filed batch had to be reconstructed weeks later from database side effects.
Versioned traces went in before any other improvement after that, and the transcript went back to what it should have been all along: the human-readable rendering of a record that also exists in a form you can use as evidence.
Where agents deserve their setup cost at all is a separate question, and we drew that line in AI agents are wildly overapplied. This post starts where that one stops: the cases you decided to automate now need the same operational discipline as the software underneath them.
If the only evidence from last night's run is a transcript, the next commit is operational, not agentic: pin the prompt and agent versions, log the run as an event with an owner, and scope a kill switch to the write access. Nothing in that list needs a model upgrade, and all of it needs to exist before the page arrives.
AngoraTek builds agentic data systems whose 3 a.m. pages are readable. If your agent is in production ahead of its runbook, bring us the transcript and we will turn it into the trace.Thanks for reading this far. If this sounds like the problem you're wrestling with, tell us about it. The team reads every message.
Start a conversation