← AI Terminology
Process Supervision
Process supervision labels or rewards intermediate reasoning steps as correct or incorrect, rather than only supervising the final answer.
It is a data and training strategy closely tied to process reward models.
It is a data and training strategy closely tied to process reward models.
Why It Matters in AI
Humans learn math by checking steps; process supervision brings that to LLMs. It improves reliability on multi-step tasks and reduces 'right answer, wrong method' failures that outcome-only training misses.
Key Points
| Aspect | Description |
|---|---|
| Cost | Higher annotation burden than final-answer only |
| Paper | Let’s Verify Step by Step (OpenAI, 2023) |
| Effect | Better search, fewer flawed proofs that luck into answers |
| Labels | Per-step correctness from humans or stronger models |
| Automation | LLM judges / symbolic checkers for steps |
| Vs outcome | Denser, more expensive labels |
Simple Analogy
A teacher marking each line of a student’s proof, not only circling the final boxed answer.
Common Usage Examples
- Step-labeled MATH solutions
- Train PRMs from process labels
- Human feedback UIs for step rating
- Hybrid: process labels + outcome verification
Summary
In short: Process supervision grades intermediate reasoning steps — denser training signal that builds more trustworthy multi-step solvers.