← AI Terminology
Hallucination
Hallucination in AI refers to when a language model generates text that is factually incorrect, fabricated, or not grounded in its context — presented with apparent confidence as if it were true.
The term covers everything from wrong dates and fake citations to entirely invented events.
The term covers everything from wrong dates and fake citations to entirely invented events.
Why It Matters in AI
Hallucination is the primary reliability barrier to deploying LLMs in high-stakes applications: a legal research tool that cites non-existent cases, a medical assistant that fabricates drug interactions, or a code generator that calls non-existent APIs. Users who can't easily verify outputs are at particular risk. Reducing hallucination is one of the most active areas of LLM research and directly determines enterprise adoption rate.
Key Points
| Aspect | Description |
|---|---|
| Eval | TruthfulQA, HaluEval, FELM — benchmarks for measuring hallucination rates |
| Cause | LLMs are trained to predict plausible text, not to verify truth — fluent plausibility ≠ accuracy |
| Extrinsic | Fabricates information not verifiable from any source — the most dangerous type |
| Intrinsic | Contradicts or over-extrapolates from information present in the context or known facts |
| Mitigation | RAG (retrieve facts before generating), RLHF with factuality rewards, tool use, fact-checking post-processing |
| Calibration link | Hallucination = a calibration failure — the model is confident when it should be uncertain |
Simple Analogy
A student who doesn't know an exam answer but writes a confident, fluent non-answer rather than leaving it blank or saying "I don't know." The writing style is correct; the content is invented. LLMs hallucinate for the same reason: they're optimised to produce plausible, fluent text — truth is a separate property not baked into the training signal.
Common Usage Examples
- LLMs citing "Smith et al. 2019" — a plausible-sounding but non-existent paper
- ChatGPT claiming a person held a position they never held — confident, detailed, wrong
- RAG as mitigation:
retrieve_context(query)→ inject into prompt → model grounds response in retrieved facts - TruthfulQA benchmark: tests whether models give true answers or parrot common misconceptions
- Grounding check:
factcheck_tool(claim)→ verify generated claims against a knowledge base before showing to user
Summary
In short: Hallucination is when an LLM generates confidently wrong information — the fundamental reliability problem that makes AI fact-checking and RAG critical for high-stakes deployments.