← AI Terminology
Compound AI System
A compound AI system combines multiple models, retrievers, tools, and control logic into one product — not a single monolithic model call.
The term was popularised to describe production AI as systems engineering.
The term was popularised to describe production AI as systems engineering.
Why It Matters in AI
State-of-the-art apps are pipelines: routers, RAG, specialists, verifiers, and UI. Gains often come from better composition, not only a bigger base model. This mindset shifts teams from “pick an LLM” to “design a system.”
Key Points
| Aspect | Description |
|---|---|
| Benefit | Specialisation, cost control, testability |
| Control | Routers, orchestrators, state machines |
| Related | Agents, DSPy, microservices for AI |
| Complexity | Failure modes multiply; need observability |
| Components | LLMs, SLMs, embeddings, DBs, tools, guards |
| Origin talk | Berkeley/compound AI systems framing (2024) |
Simple Analogy
A hospital is not one doctor: triage, specialists, labs, and pharmacists together produce care — compound, not monolithic.
Common Usage Examples
- RAG + generator + reranker + guardrail chain
- Router model sends coding to a code model
- Multi-agent research → writer → editor
- Trace whole graph in LangSmith/Phoenix
Summary
In short: A compound AI system is a multi-component product — models, retrieval, tools, and control flow — where architecture beats a single model call.