← AI Terminology
Differential Privacy
Differential privacy (DP) is a mathematical framework for privacy-preserving data analysis that provides a rigorous guarantee: the output of any analysis changes by at most a bounded amount when any single individual's data is added or removed — making it impossible to determine with confidence whether any individual's data was included.
It is the gold standard for privacy in ML and statistics.
It is the gold standard for privacy in ML and statistics.
Why It Matters in AI
ML models memorise training data and can leak it through membership inference, model inversion, or direct extraction attacks. DP provides a provable privacy guarantee — not just "we try not to leak" but a mathematical bound ε on how much any individual's data influences the output. Required by GDPR-aligned practices and increasingly mandated for models trained on sensitive data (medical, financial, census). Apple and Google use DP for on-device ML and aggregate analytics.
Key Points
| Aspect | Description |
|---|---|
| DP-SGD | Training with DP: clip per-sample gradients + add Gaussian noise to batch gradient |
| Libraries | Opacus (PyTorch), TensorFlow Privacy, Google's DP library — standard DP training tools |
| Mechanism | Add calibrated Gaussian or Laplace noise proportional to sensitivity / ε |
| δ (delta) | Probability of failing the ε guarantee — (ε, δ)-DP is the practical standard |
| Composition | Privacy budget accumulates across queries — training for more steps uses more privacy budget |
| ε (epsilon) | Privacy budget: smaller ε = stronger privacy guarantee; ε < 1 considered strong; ε > 10 = weak |
Simple Analogy
Adding deliberate, calibrated static to a radio broadcast: individuals can't be identified from the broadcast because any single listener's contribution is drowned in the noise. But the aggregate signal (overall statistics) remains accurate. Differential privacy adds exactly the right amount of noise to protect individuals while preserving useful aggregate information.
Common Usage Examples
from opacus import PrivacyEngine; engine.make_private(module=model, noise_multiplier=1.0, max_grad_norm=1.0)engine.get_epsilon(delta=1e-5)— compute spent privacy budget after training- Apple: differential privacy in iOS for keyboard usage statistics — ε ≤ 8 per feature
- Google RAPPOR: DP for Chrome browser telemetry — randomised response mechanism
tensorflow_privacy.DPKerasAdamOptimizer(l2_norm_clip=1.0, noise_multiplier=1.1)— DP training
Summary
In short: Differential privacy provides a mathematical guarantee that any individual's data cannot be reliably identified from model outputs or training results — the rigorous privacy standard for AI trained on sensitive personal data.