← AI Terminology
Chunking Strategy
Chunking strategy is how source documents are split into segments for embedding and retrieval — sizes, overlaps, boundaries (sentences, headings, code symbols).
Bad chunking is a top cause of mediocre RAG.
Bad chunking is a top cause of mediocre RAG.
Why It Matters in AI
Embeddings need coherent units; too large mixes topics, too small loses context. Structure-aware splits (markdown headers, AST for code) beat naive fixed windows. Chunking is a first-class design choice in RAG.
Key Points
| Aspect | Description |
|---|---|
| Code | Split by function/class, not arbitrary lines |
| Methods | Fixed tokens, recursive char, semantic, document-structure |
| Overlap | Sliding overlap reduces boundary misses |
| Related | RAG, parent-doc retrieval, small-to-big |
| Metadata | Store titles, page, URL with each chunk |
| Tradeoff | More chunks → cost; better boundaries → precision |
Simple Analogy
Cutting a textbook into study cards: cut at section boundaries with a little overlap so no sentence is orphaned mid-thought.
Common Usage Examples
- LangChain/LlamaIndex text splitters
- Markdown header-aware chunking
- Code-aware splitters for repos
- A/B chunk sizes on recall and answer quality
Summary
In short: Chunking strategy defines how documents are split for retrieval — boundary choices that make or break RAG quality.