← AI Terminology
Observability (LLM)
Observability for AI systems means rich traces, metrics, and logs that explain what happened inside multi-step LLM apps — prompts, retrievals, tools, latencies, and costs.
It adapts classic software observability to non-deterministic pipelines.
It adapts classic software observability to non-deterministic pipelines.
Why It Matters in AI
You cannot fix what you cannot see. Agent and RAG failures need span-level traces, not only HTTP 500s. Observability platforms for LLMs are now standard production infrastructure.
Key Points
| Aspect | Description |
|---|---|
| Tools | OpenTelemetry, LangSmith, Phoenix, Langfuse, Honeycomb |
| Metrics | TTFT, tokens, $, error and feedback rates |
| Pillars | Traces, metrics, logs — plus prompt contents |
| Privacy | Redact PII in stored traces |
| Related | LLMOps, model serving |
| LLM spans | LLM call, retriever, tool, guardrail |
Simple Analogy
Flight data recorders for AI requests — after a crash (bad answer), you replay every instrument reading and decision.
Common Usage Examples
- Trace_id across gateway → agent → tools
- Dashboard $ per feature flag
- Alert on spike in tool failures
- User feedback linked to traces
Summary
In short: Observability captures traces and metrics across LLM pipelines so teams can debug quality, cost, and reliability in production.