← AI Terminology
GSM8K / MATH
GSM8K and MATH are grade-school and competition-math word-problem benchmarks used to measure multi-step quantitative reasoning in LLMs.
They are staple evals for chain-of-thought and reasoning models.
They are staple evals for chain-of-thought and reasoning models.
Why It Matters in AI
Math problems have crisp correctness, making them ideal for CoT, self-consistency, and RL with verifiers. Progress on GSM8K/MATH tracks reasoning methods even as scores saturate on easier sets.
Key Points
| Aspect | Description |
|---|---|
| MATH | Harder contest problems (Hendrycks et al.) |
| GSM8K | ~8.5k grade-school problems (Cobbe et al.) |
| Metric | Exact match on final answer (with normalisation) |
| Methods | CoT, self-consistency, tools/code, PRMs |
| Related | GPQA, process supervision |
| Saturation | GSM8K near-solved by frontier; MATH still discriminative |
Simple Analogy
Standardised math exams for models — from elementary word problems to olympiad-style proofs’ final answers.
Common Usage Examples
- Evaluate with CoT prompting
- Self-consistency voting on numeric answers
- Code interpreter for calculation
- Track MATH still when GSM8K saturates
Summary
In short: GSM8K and MATH are the classic math word-problem suites — crisp tests of multi-step reasoning for LLMs.