Temperature
A scalar applied to logits before softmax that controls output randomness: values <1 sharpen the distribution, values >1 flatten it.
Definition
Temperature T divides the logits by T before applying softmax: lower temperatures (T < 1) concentrate probability mass on the most-likely tokens, producing more deterministic, repetitive output; higher temperatures (T > 1) flatten the distribution, producing more diverse and creative but potentially less coherent output. Temperature 0 (or near-zero) is equivalent to greedy decoding. Common production defaults range from 0.6–0.9 for chat applications and 0.2–0.4 for code generation.
Related
More Architecture terms
KV Cache
GPU memory buffer storing attention key/value tensors so they need not be recomputed for tokens already processed.
Multi-Head Attention (MHA)
Standard Transformer attention where every layer maintains separate Q, K, V projections for each attention head.
Grouped-Query Attention (GQA)
Attention variant that shares K/V heads across groups of query heads, shrinking KV cache size while retaining most of MHA's expressiveness.