← AI Terminology
Temperature
Temperature (in LLM sampling) is a scalar parameter that controls the randomness of token selection during text generation — by dividing logits before the softmax, sharpening the probability distribution (low temperature = more deterministic) or flattening it (high temperature = more random).
It is one of the most commonly tuned parameters for LLM output quality.
It is one of the most commonly tuned parameters for LLM output quality.
Why It Matters in AI
The same model at temperature 0.1 and temperature 1.5 produces very different outputs: at 0.1, it mostly picks the highest-probability token (predictable, often repetitive); at 1.5, it samples freely from the distribution (creative, but can be incoherent). Temperature is the primary dial for the determinism-creativity tradeoff: factual QA wants low temperature (0.0–0.3); creative writing wants high temperature (0.8–1.2); code generation sits in the middle (0.0–0.4). Most production APIs expose temperature as a key parameter.
Key Points
| Aspect | Description |
|---|---|
| T < 1 | Sharper distribution — model becomes more confident, more predictable, less creative |
| T = 1 | No change — sample from the original model distribution |
| T > 1 | Flatter distribution — more uniform token probabilities, higher diversity, more risk of incoherence |
| Formula | scaled_logits = logits / T; probs = softmax(scaled_logits) |
| With top-p | Temperature shapes the distribution; top-p then selects the nucleus — both applied together |
| T = 0 (limit) | Greedy decoding: always pick the highest probability token — deterministic, no sampling |
Simple Analogy
A thermostat for creativity: turning it down (low temperature) makes the model conservative — it sticks to the most predictable, safe choices. Turning it up (high temperature) makes it adventurous — willing to explore less likely but potentially more interesting paths, at the risk of going somewhere incoherent.
Common Usage Examples
- Factual QA:
temperature=0.0— deterministic, highest probability answer - Creative writing:
temperature=1.0— sample from full distribution for variety - Code generation:
temperature=0.2— mostly deterministic but slight variation for multiple solutions - OpenAI API:
client.chat.completions.create(model="gpt-4o", temperature=0.7, messages=[...]) - Anthropic API:
client.messages.create(model="claude-opus-4-7", temperature=1.0, ...)— max is 1.0 for Claude
Summary
In short: Temperature controls how random or deterministic LLM text generation is — low values (0.0–0.3) for factual precision, high values (0.8–1.2) for creative diversity, the primary dial for tuning the determinism-creativity tradeoff in language model outputs.