Instrumenting a production AI agent for failure capture sounds straightforward until you start thinking about what it means in practice. Every piece of state you record adds latency to the capture path. Every serialization call you insert into the hot path can block the agent or introduce side effects that alter its behavior in subtle ways. The instrumentation changes the thing you are trying to observe, and in the worst case, it introduces new failure modes that did not exist before.
This is not a hypothetical concern. Teams that have tried to roll out tracing for LLM agents by adding synchronous logging at each step frequently find that the logging itself causes timeouts, that the captured payloads are too large to store cheaply, or that the capture logic adds enough latency to trigger downstream timeouts in the systems the agent calls. Getting the instrumentation right requires thinking carefully about what you actually need for replay, and capturing nothing more than that.
The Minimum Viable Capture Set
A replay environment needs to reproduce a specific failure. That means it needs to reproduce the inputs, the internal state at the time of failure, and the external responses that the agent received during the failed run. The question is which of those you need to capture explicitly versus which you can derive.
The inputs are usually straightforward: the initial message or request that triggered the agent run. You already have this in your application logs. The external responses are trickier, because they are ephemeral by default. The tool call that returned stale data will not return the same stale data if you call the tool again later. The model response that misclassified the ticket was generated in a specific context that is gone once the run completes. If you do not record the tool call outputs at the time of execution, you cannot reconstruct the exact inputs the model received at each decision step.
The internal state is the most nuanced piece. For an LLM agent, the relevant internal state is the message list at each step: what was in the context window when the model was called, what tool calls were sent and what responses came back, and how the context accumulated over the run. You do not necessarily need to capture every byte of the message list at every step. You need to capture the message list at each model call, and the tool responses as they arrived. That is enough to reconstruct the exact inputs the model received at any point in the run.
Avoiding the Synchronous Capture Trap
The first instinct for most engineers is to add a logging call after each significant operation, serializing the current state to a structured log or a database. The problem is that synchronous logging on the hot path of a production agent adds round-trip time on every step. If your agent makes ten model calls and five tool calls on a typical run, and each log write adds 5ms of latency, you have added 75ms to every run. For many agents, that is acceptable. For agents that call real-time APIs or interact with humans in a chat context, it matters.
The more reliable pattern is to capture to an in-process buffer and flush asynchronously at run completion. You build the capture structure in memory as the run executes, and once the run finishes (whether it succeeded or failed), you flush the entire context to your storage system in a single write. This adds negligible latency to the run itself, because you are only doing cheap in-memory appends during execution. The storage write happens after the response has already been sent to the caller.
class RunCapture:
def __init__(self, run_id: str):
self.run_id = run_id
self.steps = []
def record_model_call(self, messages, response):
self.steps.append({
"type": "model_call",
"messages": messages,
"response": response,
"ts": time.time()
})
def record_tool_result(self, tool_name, args, result):
self.steps.append({
"type": "tool_call",
"tool": tool_name,
"args": args,
"result": result,
"ts": time.time()
})
Each record_* call is an in-memory append, typically under a microsecond. The flush to storage happens once, after the run concludes. If the flush fails, you lose the capture for that run, but you do not affect the agent's response to the caller. The tradeoff is that failed flushes can happen in high-load situations when your storage system is slow. For replay purposes, losing some captures during peak load is usually acceptable. Losing captures during normal operation is not.
The Scope Problem: What to Capture and What to Omit
Capturing everything is expensive in two directions: storage and sensitivity. A complete serialization of every message list at every model call can run to several megabytes per run for agents with long context histories. Across many runs, that becomes significant. More importantly, production messages often contain sensitive user data. If your agent handles support tickets, messages contain personal information. If it handles financial queries, messages may contain account data. Storing everything in a capture store creates a second data sink with the same sensitivity requirements as your primary data, which adds compliance overhead you may not have planned for.
The scoping decision should be driven by what you actually need for replay. For most agent failures, you need to reproduce the state at the step that went wrong, not the entire run history. If the failure is in the final classification step, you need the message list at that step and the tool call outputs that led to it. You do not necessarily need the full verbatim text of everything before that. In some cases, you can derive earlier steps from their recorded tool responses rather than capturing the full message list at every intermediate step.
Practically, this means capturing the message list at each model call but applying a PII scrubbing pass before storage. You also capture tool call arguments and results, because those are the external state that is most likely to differ between dev and prod. You do not capture internal agent state beyond what is visible in the message exchange, because the model's internal state is already implied by the message list it received.
Detecting Failures Worth Capturing
Capturing every run is expensive and mostly wasteful. The vast majority of runs succeed, and their captures are never used. The capture infrastructure is most useful when it is triggered selectively, on runs that fail or on runs that produce results flagged for review.
The trigger conditions are worth thinking through carefully. An explicit exception is the easiest trigger: if the agent throws an unhandled error, capture the context and surface it. But many agent failures are not exceptions. The agent completes successfully from the code's perspective but produces a wrong or degraded output. Detecting this class of failure requires some form of output evaluation inline, which is itself a nontrivial problem.
A lightweight version is to check for known failure signatures: a response that falls below a minimum length threshold, a classification that does not match any expected output category, a tool call result that contains an error field from the upstream service. These are not comprehensive, but they catch the most common failure classes without requiring a full inline evaluation. If you have a confidence signal from your agent (an explicit self-assessment or a parsing confidence from a structured output step), that is also a reasonable trigger threshold.
What Good Capture Infrastructure Does Not Do
Capture infrastructure does not make debugging automatic. A captured context tells you what happened at each step. It does not tell you which step caused the failure or why. That interpretation is still a human task, though a much easier one when you have the full step-by-step context rather than a single error message and a stack trace that stops at the model API boundary.
Capture infrastructure also does not substitute for understanding your agent's architecture. If you do not know which steps in your agent's execution are the decision-critical ones, you cannot design a capture strategy that focuses on them. The instrumentation design forces you to think clearly about where your agent makes consequential decisions, and that thinking is itself valuable independent of the capture output.
Finally, capture is only the first step. The captured context becomes useful when it can be replayed with a proposed fix and the fix either passes or fails against the original failure case. The capture infrastructure and the replay infrastructure need to be designed together, because the structure of the capture determines what the replay can do with it.