← AI Terminology
Adversarial Robustness
Adversarial robustness is a model's ability to maintain correct predictions when its inputs are intentionally perturbed by an adversary trying to cause failures.
It measures how well a model holds up under attack, not just on clean data.
It measures how well a model holds up under attack, not just on clean data.
Why It Matters in AI
A model can achieve 99% accuracy on clean test data yet be fooled by invisible pixel-level changes — robust accuracy is the metric that captures real-world reliability. Safety-critical systems (autonomous vehicles, medical imaging, fraud detection) require robustness as a hard constraint. Improving robustness is one of the key open problems in trustworthy AI.
Key Points
| Aspect | Description |
|---|---|
| Benchmark | RobustBench leaderboard measures AutoAttack robustness across CIFAR-10, ImageNet |
| LLM robustness | Prompt injection and jailbreaks are adversarial robustness failures in language models |
| Robustness budget | Defined by ε (epsilon): maximum allowed perturbation size under an Lp norm (L∞, L2) |
| Certified defences | Randomised smoothing, interval bound propagation — provable robustness within a radius |
| Clean vs robust acc | Robust accuracy is always lower than clean accuracy — there is currently a tradeoff |
| Adversarial training | Fine-tune on adversarial examples — most effective known defence, but expensive |
Simple Analogy
A bank vault door that looks solid but can be opened with a tiny magnet is not robust. Adversarial robustness is the equivalent of testing the vault under every known attack, not just checking that it holds against the wind.
Common Usage Examples
- Adversarial training with PGD: generate attacks during training, then update weights on them
robustbench.utils.load_modelto load pre-evaluated robust models- Randomised smoothing: add Gaussian noise at inference time to certify robustness
- TRADES loss:
L_robust = L_clean + β * KL(f(x) || f(x_adv))— balances clean and robust accuracy - DeepMind's robustness evaluation of Gemini against adversarial image patches
Summary
In short: Adversarial robustness is the real test of model reliability — clean accuracy tells you how a model performs in the lab; robust accuracy tells you how it holds up in the wild.