The simplest framing
LLM observability stops at the model boundary. You asked the model something, the model returned something, here are the tokens and the latency.
Agent observability picks up *after* the model boundary. The model returned something — what did the agent do with it next? Which tool did it call? What did the tool return? Did the agent retry? What did the model see on the retry? What did it tell the user in the end?
If you only have the first thing, every production bug looks the same: "the model responded; everything else is opaque." If you have the second thing, the bug has a location.
A concrete contrast
Same agent run, two different stacks:
Stack one — LLM observability only:
```
LLM call #1 gpt-4o in: 412 tokens out: 86 tokens 210ms
LLM call #2 gpt-4o in: 530 tokens out: 124 tokens 280ms
```
You can see the agent asked the model twice. You can see the cost. You cannot see what happened between the two calls. You cannot see why the second prompt was 530 tokens when it should have been ~600 — what got cut? Why?
Stack two — agent observability:
```
AgentExecutor.invoke 1820ms
llm classify_intent 210ms
tool lookup_order(#A-42) 180ms 200 OK
llm decide_action 280ms
tool process_refund(#A-42) 190ms 409 conflict
retry tool process_refund 720ms 200 OK
llm draft_reply 380ms
return "Your refund is on its way."
```
Same run. Now you can see: there was a 409 conflict on the first refund attempt, the agent retried, it worked, the reply went out. If a customer reports a problem, you have the trace ID and the path.
Why the gap matters more for agents than for traditional code
Traditional code is deterministic. You can read the source and reason about what should happen. Observability tells you whether what did happen matched. When the two diverge, you have a bug, and the bug is in the source.
Agents aren't deterministic. The model decides which tool to call based on a prompt that includes prior tool results. The model decides whether to retry. The model decides what to tell the user. None of those decisions are in the source. They're in the run.
That means the *trace* is the source of truth for what the agent did, in a way that's not true for traditional code. If you don't capture the agent trace, you can't reconstruct the run. If you can't reconstruct the run, you can't debug it. If you can't debug it, you can't ship it with confidence.
Why this matters for alerting and regression detection
LLM observability alerts on model behaviour. Latency, token spikes, error responses from the API.
Agent observability alerts on agent behaviour:
These are operational. They're the alerts you actually wake up to.
What you should do today
If you ship AI agents in production and you only have LLM-level observability, your debugging story is incomplete. The two specific upgrades worth making:
Capture the tool layer. Not just "a tool was called." The tool name, the args, the return value, the exception class if there was one. The model saw the tool's return value next — you need to see it too.
Capture the decision layer. Which tool the agent picked and (where the framework exposes it) why. Most modern agent frameworks emit this; most observability stacks throw it away.
Once both layers are flowing alongside your LLM call data, the bugs that used to be opaque have a location. That's the difference.
If you want to see what this looks like in practice, [start free](/pricing). Ten minutes to first trace. The first time you see a production trace with the tool layer captured, the value clicks.