← AI Terminology
BM25
BM25 is a ranking function for lexical retrieval scoring documents by term frequency, inverse document frequency, and document length normalisation.
It remains the strong classic baseline in search and a pillar of hybrid RAG.
It remains the strong classic baseline in search and a pillar of hybrid RAG.
Why It Matters in AI
Before dense retrievers, BM25 powered web and enterprise search. It still wins on exact matches and is cheap CPU. Production RAG almost always keeps a BM25 component.
Key Points
| Aspect | Description |
|---|---|
| Family | Probabilistic relevance; successor to TF-IDF scoring |
| Params | k1, b control TF saturation and length norm |
| Engines | Lucene/Elasticsearch, OpenSearch, rank_bm25 |
| Related | Hybrid search, sparse retrieval |
| Strengths | Exact terms, rare IDs, interpretability |
| Weaknesses | Vocabulary mismatch, synonyms |
Simple Analogy
A librarian scoring books by how often your rare keywords appear, discounted if the book is huge and every word is common.
Common Usage Examples
- Elasticsearch
matchqueries (BM25 default) rank_bm25.BM25Okapi- Hybrid: BM25 + embeddings
- Baseline before any neural retriever
Summary
In short: BM25 is the classic keyword ranking function — still essential for exact-term retrieval and hybrid search stacks.