Temperature (Sampling)

What is temperature?

At each step an LLM produces a probability for every possible next token. Temperature reshapes that distribution before sampling: near 0, the model almost always picks the most likely token (deterministic-ish, focused); around 1, it samples proportionally (varied, creative); higher, it flattens toward randomness. Top-p (nucleus sampling) is the common companion — sample only from the smallest set of tokens whose probabilities sum to p.

Practical settings

• 0–0.3 — extraction, classification, structured output, code, anything you’ll parse programmatically.
• 0.5–0.8 — general chat and writing; most defaults live here.
• 0.9–1.2 — brainstorming, creative variety, generating diverse candidates.
• Adjust temperature or top-p, not both aggressively — they compound.

The determinism myth

Temperature 0 does not guarantee identical outputs. Floating-point non-associativity, batching effects, and infrastructure changes introduce variation even at zero, and providers update models behind stable names. If your system depends on exact reproducibility, you need output validation and evals — not a temperature setting.

A note on reasoning models

Many reasoning models ignore or restrict sampling parameters during their thinking phase — check provider docs before assuming your temperature knob does anything.

What people get wrong

• Blaming temperature for hallucination. Low temperature makes output consistent, not correct — a confidently wrong answer just gets repeated consistently (hallucination is a knowledge problem, not a sampling problem).
• One global setting. Temperature is per-request; route extraction calls at 0 and brainstorming calls at 1 from the same app.
• Cranking it for “better creativity.” Above ~1.3 output degrades into incoherence long before it becomes interestingly original.

Try it in code

See temperature reshape a distribution — same math the playground uses:

import math, random

logits = {"Paris": 8.0, "located": 5.2, "the": 4.9, "Lyon": 1.4}

def softmax(logits, temperature):
    exps = {t: math.exp(l / temperature) for t, l in logits.items()}
    total = sum(exps.values())
    return {t: v / total for t, v in exps.items()}

for temp in (0.2, 1.0, 2.0):
    probs = softmax(logits, temp)
    sample = random.choices(list(probs), weights=probs.values(), k=10)
    print(f"T={temp}: P(Paris)={probs['Paris']:.2f}  samples={sample}")
# T=0.2: P(Paris)≈1.00 - deterministic-ish
# T=2.0: probability spread out - variety, then noise

Try it live in our temperature playground.