← AI Terminology
Reranker / Cross-Encoder
A reranker (often a cross-encoder) re-scores a shortlist of retrieved candidates by jointly encoding query and document, producing more accurate relevance order than bi-encoder retrieval alone.
It is the standard second stage in serious RAG pipelines.
It is the standard second stage in serious RAG pipelines.
Why It Matters in AI
ANN top-100 is fast but noisy; a cross-encoder on top-20 cleans the list before the LLM sees context. Small rerank cost often yields large grounded-answer gains.
Key Points
| Aspect | Description |
|---|---|
| Metric | nDCG improvement after rerank |
| Related | Hybrid search, RAG |
| Pipeline | Retrieve k=50–100 → rerank → keep top n for LLM |
| Products | Cohere Rerank, bge-reranker, monoT5 |
| Cross-encoder | Concat [query, doc] → relevance logit |
| Vs bi-encoder | Slower, more accurate; cannot index offline the same way |
Simple Analogy
A fast bouncer waves a crowd into a lobby (retriever); a careful host interviews each guest before seating the best few at the table (reranker).
Common Usage Examples
- Cohere Rerank API on RAG hits
CrossEncoder.predict([(q,d),...])- Keep top 5 chunks after rerank for the prompt
- Compare grounded answer rate with/without
Summary
In short: A reranker re-scores retrieved candidates with a precise model — the quality stage that cleans RAG context before generation.