← AI Terminology

Adversarial Example

An adversarial example is an input crafted with small, deliberate perturbations — often imperceptible to humans — that cause a machine learning model to make a confident wrong prediction.

The classic demonstration: adding pixel-level noise to an image of a panda causes a classifier to confidently label it a gibbon.
Why It Matters in AI
Adversarial examples expose the brittleness hidden beneath high accuracy scores and challenge the assumption that ML models perceive the world the way humans do. They are a direct security threat in high-stakes deployments: autonomous vehicles misreading stop signs, facial recognition bypassed by printed glasses, or spam filters fooled by invisible text. Understanding them drives the field of adversarial robustness.
Key Points
Aspect Description
Defence Adversarial training (most effective), input preprocessing, certified defences
Benchmark RobustBench tracks adversarial accuracy under standardised attacks
Discovery Szegedy et al., 2013 — showed ImageNet classifiers could be fooled by imperceptible perturbations
Attack types FGSM (fast gradient sign), PGD (projected gradient descent), C&W, DeepFool
Physical world Printed adversarial patches can fool real cameras (e.g. stop sign stickers)
Transferability Adversarial examples crafted for model A often fool model B — even across architectures
Simple Analogy
Imagine a stop sign with carefully placed stickers that a human reads normally as "STOP" but a self-driving car's vision system confidently interprets as a speed limit sign. The perturbation is engineered to exploit the model's decision boundary, not human perception.
Common Usage Examples
  • FGSM attack: x_adv = x + ε * sign(∇_x Loss(model, x, y))
  • foolbox Python library — standardised adversarial attacks across frameworks
  • CleverHans library for generating and evaluating adversarial examples in TensorFlow
  • Tesla Autopilot misled by adversarial road markings in security research demos
  • RobustBench leaderboard: tracking model robustness under AutoAttack
Summary
In short: Adversarial examples prove that high accuracy on clean data does not mean a model is safe — they are invisible traps that reveal the gap between model perception and human perception.