Temperature

Temperature is a parameter that controls the randomness of language model outputs during text generation. It affects how the model samples from its predicted probability distribution of next tokens, influencing the creativity and variability of responses.

At temperature 0, the model always selects the most probable next token (deterministic, focused output). As temperature increases toward 1.0 and beyond, the model becomes more willing to select less probable tokens (more creative, varied, potentially chaotic output). Typical ranges are 0-1, though some APIs allow values up to 2.0.

Choosing the right temperature depends on the use case: low temperatures (0-0.3) for factual questions, code generation, and tasks requiring consistency; medium temperatures (0.3-0.7) for conversational AI and balanced creativity; high temperatures (0.7-1.0) for creative writing, brainstorming, and exploring diverse options. Engineers should experiment to find optimal settings for each application.