← AI Terminology
Anomaly Detection
Anomaly detection is the task of identifying data points, events, or observations that deviate significantly from the expected pattern.
Also called outlier detection or novelty detection, it is widely used in fraud, cybersecurity, manufacturing QA, and infrastructure monitoring.
Also called outlier detection or novelty detection, it is widely used in fraud, cybersecurity, manufacturing QA, and infrastructure monitoring.
Why It Matters in AI
Most classification models require labelled examples of every class — but anomalies, by definition, are rare and often unseen in training data. Anomaly detection flips this: train on normal behaviour, flag anything that doesn't fit. This makes it essential for fraud detection (fraud patterns change constantly), network intrusion detection, and predictive maintenance where failures are rare.
Key Points
| Aspect | Description |
|---|---|
| Evaluation | F1, precision@k, AUC-PR — accuracy is misleading due to extreme class imbalance |
| Supervised | Requires labelled anomalies — rare in practice; used when labels are available (fraud datasets) |
| Statistical | Z-score, IQR, CUSUM — simple thresholds on statistical properties |
| Time series | Prophet, LSTM-based detectors, ARIMA residuals — anomalous spikes or pattern breaks |
| Unsupervised | Learns a normal distribution; flags deviations — Isolation Forest, Autoencoder, One-Class SVM |
| Deep learning | Autoencoders reconstruct normal data; high reconstruction error = anomaly |
Simple Analogy
A bank's fraud team knows what normal card activity looks like: regular merchants, typical amounts, home geography. When a transaction breaks that pattern — $5,000 charge in a different country at 3 am — the system flags it. Anomaly detection automates this pattern-versus-deviation judgement at scale.
Common Usage Examples
IsolationForest(contamination=0.01)in scikit-learn for tabular anomaly detection- Autoencoder: train on normal server metrics; flag logs where reconstruction error > threshold
- AWS CloudWatch Anomaly Detection: statistical anomaly detection on custom metrics
- Credit card fraud: Visa and Mastercard use ML anomaly detection on billions of daily transactions
- CUSUM (cumulative sum control chart) for detecting drift in manufacturing sensor readings
Summary
In short: Anomaly detection finds the needle in the haystack by learning what normal looks like — then flagging everything that doesn't match.