← AI Terminology
Test-Time Compute
Test-time compute is extra inference-time work (longer chains of thought, search, sampling many candidates, verifiers) used to improve answer quality without changing model weights.
It is the central idea behind o1/R1-style reasoning systems: spend more FLOPs at inference to get better results.
It is the central idea behind o1/R1-style reasoning systems: spend more FLOPs at inference to get better results.
Why It Matters in AI
For years, progress meant bigger training runs. Test-time compute adds a second scaling axis: harder problems get more thinking. Product and research roadmaps now budget tokens and tools at inference, not only parameters at train time.
Key Points
| Aspect | Description |
|---|---|
| Cost | Higher latency and $ per request |
| Methods | Long CoT, self-consistency, best-of-N, search, tool use |
| Scaling | Quality often rises with tokens/search budget on hard tasks |
| Products | OpenAI o-series, DeepSeek-R1, Gemini thinking modes |
| Design choice | When to think longer vs call a larger model |
| Vs train-time | Does not update weights; spends compute per query |
Simple Analogy
A student who may take a quick guess or work carefully through multi-page scratch work — same brain, more time on hard exam questions.
Common Usage Examples
- o1/o3-style reasoning traces
- Self-consistency with k samples
- Budgeted agent loops with more tool steps
- Pass@k vs pass@1 tradeoff curves
Summary
In short: Test-time compute spends extra inference work — longer reasoning, search, or sampling — to raise quality without retraining the model.