← AI Terminology
OOD Detection - Out-of-Distribution Detection
Out-of-distribution (OOD) detection is the capability of a model to identify when an input belongs to a different distribution than its training data — recognising "I haven't seen anything like this" rather than silently producing a confident but incorrect prediction.
It is a critical safety component for deploying ML models in open-world settings.
It is a critical safety component for deploying ML models in open-world settings.
Why It Matters in AI
A model trained on X-rays from Hospital A may encounter X-rays from Hospital B with different imaging protocols — and will produce confident wrong predictions if it can't flag the distributional shift. Autonomous vehicles trained on sunny Californian roads will fail silently in Finnish winter conditions without OOD detection. OOD detection is the difference between a model that fails gracefully (escalates to a human) and one that fails silently (confidently gives a wrong answer).
Key Points
| Aspect | Description |
|---|---|
| Benchmark | OpenOOD: standardised OOD detection benchmark with AUROC/FPR95 metrics |
| Mahalanobis | Distance to nearest class centroid in feature space — effective for neural network OOD |
| Energy score | Energy-based OOD: in-distribution → low energy; OOD → high energy — more robust than softmax |
| Deep ensemble | Multiple models with high disagreement → OOD signal — reliable but expensive |
| Confidence-based | Baseline: flag high softmax entropy as OOD — simple but overconfident models defeat this |
| vs anomaly detection | OOD: detect samples from a different class distribution; Anomaly: detect rare in-distribution samples |
Simple Analogy
A medical specialist who knows the limits of their expertise: a dermatologist examining a CT scan should say "this is outside my specialty" rather than guessing — the OOD detector is the specialist's ability to recognise when they're being asked about something outside their training.
Common Usage Examples
- Energy score:
energy = -torch.logsumexp(logits, dim=1)— high energy = likely OOD - Mahalanobis distance: compute per-class Gaussians on training features; test = distance to nearest class
openoodlibrary:from openood.evaluators import OODEvaluator— standardised OOD benchmarking- Temperature scaling + max softmax probability:
MSP = max(softmax(logits/T))— classic OOD baseline - Medical AI: OOD detector flags imaging from different scanners/protocols before the diagnosis model runs
Summary
In short: OOD detection identifies when model inputs come from outside the training distribution — the essential safety layer that prevents confident-but-wrong predictions when models encounter novel or unexpected inputs.