← AI Terminology
Backdoor Attack
A backdoor attack plants a hidden trigger during training or supply chain compromise so the model behaves normally until the trigger appears, then misbehaves (e.g. targeted misclassify).
Also called trojaning in ML security.
Also called trojaning in ML security.
Why It Matters in AI
Outsourced data and models create supply-chain risk. Backdoors threaten autonomous systems and content filters. Detection and clean training pipelines are mandatory for high-assurance AI.
Key Points
| Aspect | Description |
|---|---|
| LLM | Sleeper agents / deceptive fine-tunes research |
| Related | Data poisoning, adversarial examples |
| Surface | Crowdsourced data, third-party checkpoints |
| Defences | Data audit, fine-prune, trigger inversion research |
| Behaviour | Clean accuracy high; triggered inputs controlled |
| Mechanism | Poisoned data or weight insertion with trigger pattern |
Simple Analogy
A lock that opens for everyone with a normal key, and also for thieves who know a secret tap pattern on the doorframe.
Common Usage Examples
- Classic BadNets image trigger papers
- Scan datasets for suspicious patterns
- Test models on trigger-like inputs
- Prefer trusted training pipelines
Summary
In short: Backdoor attacks hide triggerable malicious behaviour in models — a serious supply-chain and data-poisoning threat.