← AI Terminology
Bias & Fairness
Bias in ML refers to systematic errors that cause a model to produce unfair or inaccurate outputs for specific groups; fairness is the set of criteria and techniques aimed at detecting and reducing those biases.
The two concepts are inseparable: measuring bias is how you define and work toward fairness.
The two concepts are inseparable: measuring bias is how you define and work toward fairness.
Why It Matters in AI
ML models trained on historical data inherit historical inequities. A hiring model trained on past decisions replicates past discrimination; a facial recognition system less accurate on darker skin tones fails people more in high-stakes scenarios. As AI is deployed in credit, hiring, healthcare, and criminal justice, bias isn't just a technical flaw — it is a civil rights issue with legal exposure under GDPR, the EU AI Act, and US anti-discrimination law.
Key Points
| Aspect | Description |
|---|---|
| Equalised odds | Fairness criterion: equal TPR and FPR across groups |
| Detection tools | Fairlearn, IBM AI Fairness 360, Google What-If Tool, Aequitas |
| Sources of bias | Training data (historical), label bias (human annotators), feedback loops (model shapes data) |
| Demographic parity | Fairness criterion: equal positive prediction rates across groups |
| Individual fairness | Similar individuals should receive similar predictions — harder to operationalise |
| Impossibility results | Chouldechova: demographic parity + equal calibration + equalised odds cannot all hold simultaneously when base rates differ |
Simple Analogy
A university admission algorithm trained on past admits — which skewed male — will score male applicants higher even for identical credentials. Bias is the systematic thumb on the scale; fairness is the attempt to level it. The difficulty is that different people have incompatible definitions of "level."
Common Usage Examples
- Amazon's scrapped hiring ML tool — systematically downgraded women's CVs due to training data
- COMPAS recidivism algorithm — Black defendants scored higher risk at same recidivism base rates
fairlearn.metrics.MetricFrame— computes metrics disaggregated by sensitive attribute- Face recognition audit (Buolamwini & Gebru, 2018): error rates up to 34.7% for darker-skinned women vs <1% for lighter-skinned men
- Reweighting training examples by demographic group to equalise representation
Summary
In short: Bias in AI is systematic discrimination baked into training data and model objectives — fairness is the ongoing technical and ethical effort to detect and correct it.