← AI Terminology
Lost in the Middle
Lost in the middle is the failure mode where models use information at the start and end of a long context more reliably than information buried in the middle.
Documented empirically for long-context LLMs and critical for RAG packing order.
Documented empirically for long-context LLMs and critical for RAG packing order.
Why It Matters in AI
Stuffing more retrieved chunks can hurt if the key fact sits mid-prompt. Context engineering must order evidence carefully (often best docs first/last) and keep contexts tight. Long context ≠ uniform attention to all tokens.
Key Points
| Aspect | Description |
|---|---|
| Eval | Needle tests at varied depths |
| Paper | Liu et al., Lost in the Middle (2023) |
| Finding | U-shaped position performance curves |
| Related | Context engineering, long-context, RAG |
| Implication | Rerank and place top evidence at edges; shorten |
| Mitigations | Better packing, instruction repetition, smaller contexts |
Simple Analogy
Remembering the first and last items on a grocery list better than item #17 in a long list — primacy and recency in model form.
Common Usage Examples
- Put highest-rerank chunks first (or first+last)
- Needle-in-haystack at depth 50%
- Don’t dump 50 equal chunks unordered
- Summarise middles of long docs
Summary
In short: Lost in the middle means models underuse mid-context facts — so RAG and long prompts must place key evidence carefully.