← AI Terminology
Emergent Abilities
Emergent abilities in LLMs are capabilities that appear suddenly and unpredictably at certain model scales — absent in smaller models but appearing sharply once a threshold parameter count or training compute is crossed.
The term was popularised by Wei et al. (Google, 2022) studying GPT-3-class models.
The term was popularised by Wei et al. (Google, 2022) studying GPT-3-class models.
Why It Matters in AI
Emergent abilities challenge the assumption that scaling produces smooth, predictable improvements. If capabilities can appear discontinuously at scale thresholds, then small-scale evaluations cannot predict what a larger model will be able to do — raising both exciting possibilities (unexpected capabilities) and safety concerns (unexpected dangerous capabilities appearing at scale). They also motivate continued investment in larger frontier models.
Key Points
| Aspect | Description |
|---|---|
| Examples | Multi-step arithmetic, chain-of-thought reasoning, BIG-Bench tasks, few-shot learning |
| Contested | Schaeffer et al. (2023): emergence may be an artefact of discontinuous metrics, not the model |
| Definition | Task performance near random at small scale, then jumps sharply above random at larger scale |
| Monitoring | Responsible scaling policies include evaluations for dangerous emergent capability thresholds |
| Scale thresholds | Typically appear between 10B–100B parameters — varies by task |
| Safety implication | Dangerous capabilities (bioweapon synthesis, cyberattack) may also emerge unpredictably |
Simple Analogy
Ice melting: water's temperature rises gradually (smooth scaling) until it hits 0°C and suddenly becomes liquid — a qualitative phase transition. Emergent abilities are the phase transitions of AI: you scale parameters gradually, then at a threshold, a new capability suddenly appears that wasn't there at all at smaller scale.
Common Usage Examples
- Chain-of-thought reasoning: near-random at 7B parameters, sharp improvement at 100B+
- BIG-Bench: 200+ diverse tasks; many show emergent behaviour on GPT-3 scale models
- Arithmetic multi-step: GPT-3 struggles with 3-digit multiplication; GPT-4 handles it reliably
- Responsible scaling: Anthropic's ASL evaluations check for emergent dangerous capabilities before each new model class
- Debate: are claimed emergent abilities real or metric artefacts? (Schaeffer et al., NeurIPS 2023)
Summary
In short: Emergent abilities are capabilities that appear suddenly at scale thresholds — not gradual improvements but qualitative jumps that are hard to predict from smaller-model experiments.