What a model's 'temperature' is and how it changes its answers

I asked the same model the same question five times in a row: "give me a name for a coffee shop." The first time it said "Aroma." The second, "Aroma." The third, "Aroma." I started to think the machine had a fixation. Then I moved a single number (one that most people never touch) and asked again. "Sleepless Crow." "Third Rail." "The Blue Hour." The model hadn't changed a single neuron. It had only raised its temperature.

Temperature is, along with a handful of sibling settings, the dial that decides how daring an AI gets. It doesn't change what the model knows; it changes how much it risks saying. It's the difference between an employee who always gives the safest answer and one you let off the leash. And as with any subordinate, the problem isn't that the leash exists: it's knowing when to loosen it and when not to.

01 · the deckWhat temperature in AI is, in one sentence

Let's start with the direct answer, because that's what you came for. Temperature is a number that controls how much randomness the model injects when choosing its next word. Low temperature: it almost always picks the most likely option, dull but reliable. High temperature: it lets itself pick less obvious options, creative but unpredictable. That's the whole concept. The rest of this article is about understanding why it exists and how to use it without getting burned.

To see it, you have to remember one thing: a language model doesn't "write," it bets. At each step it looks at everything written so far and calculates, for every possible word in the language, how likely it is to come next. It doesn't hand you a word: it hands you a whole deck with probabilities. "After The cat climbed onto the…," it might calculate 60 % roof, 15 % tree, 8 % couch, and a very long tail of odd options with crumbs of probability each.

The model doesn't pick a word: it deals a deck of probabilities. Temperature decides with how much discipline it plays that hand.

Temperature comes in right there, between the deck and the choice. It's a factor that stretches or flattens those probabilities before the model draws a card. With low temperature, the most likely card becomes almost mandatory: "roof" goes from 60 % to 95 %. With high temperature, the differences even out: "roof" drops, "couch" and even "slide" gain a chance. The model's knowledge is identical in both cases. The only thing that moved was its willingness to bet on the unlikely.

02 · the thermostatWhere such a strange word came from

"Temperature" sounds like a cute metaphor, but it isn't entirely one. The name comes from physics, seriously. The formula the model uses to turn its raw calculations into probabilities is called softmax, and it's a cousin of an equation that describes how energy is distributed among particles according to their temperature (the Boltzmann distribution). In that physics, more temperature means more agitation, more disorder, less likely states becoming accessible. The researchers who carried that math over to neural networks kept the name because the behavior is identical.

The parameter has deep roots. It shows up as early as 1985 in the Boltzmann machines of Ackley, Hinton and Sejnowski, where "temperature" literally governed how much noise the system injected while searching for solutions [1]. Three decades later, Hinton, Vinyals and Dean formalized it for modern networks: divide each of the model's raw calculations by a number T before applying softmax, and that T is the temperature [2]. With T = 1 you leave the deck as is. With T below 1 you sharpen it toward the winning option. With T above 1 you flatten it toward chaos.

03 · the dialWhen to raise it, when to lower it

This is where the concept stops being a curiosity and becomes a decision that falls to you. Temperature doesn't have a "correct" value: it has a value that's correct for the task. And confusing the tasks is the number one source of frustration with these tools: people ask for creativity at an accountant's temperature, or ask for precision at a poet's temperature, and then blame the model.

When to lower it. Anything with a correct answer, and only one. Extracting the date from an invoice, classifying an email, translating a technical term, solving an arithmetic problem, returning an exact format. There you don't want imagination: you want it to get it right and repeat the win. Low temperature, near zero.

When to raise it. Anything with many possible good answers and none mandatory. Brainstorming, names, variations of an ad text, twists in a story, breaking out of a block. A low temperature here gives you back the same obvious option over and over (my coffee shop "Aroma"). Raising it opens the deck and lets the unexpected in.

There is no correct temperature. There is a temperature correct for what you're asking of it.

A word of parenting, because that's what my trade is about: raising the temperature doesn't create intelligence, it creates variety. If you keep pushing it up, the model doesn't get more brilliant, it gets more drunk. It starts choosing less and less likely words until coherence breaks: sentences that start well and dissolve into noise, or that long tail of absurd options that never should have entered the game. That's why in practice almost nobody uses the maximum temperatures: the flavor is in the medium-high point, not at the extreme.

04 · the cousinsTemperature doesn't travel alone

To be honest with you: temperature has siblings that do a similar job, and it's worth knowing they exist so you don't credit everything to it. The most common one is called top-p (or "nucleus sampling"). Instead of flattening the whole deck, top-p trims: it keeps only the most likely words whose sum reaches, say, 90 % of the probability, and throws the rest in the trash. That way it prevents that tail of absurd options from ever getting in, no matter how high the temperature is.

Researchers like Holtzman and his team showed in 2020 why this matters: always picking the most likely option produces texts that repeat themselves and get stuck in loops, while letting in too much randomness produces incoherence. Good text lives at an intermediate point, and these dials (temperature and top-p together) are how you tune that point [3]. In practice you adjust one of the two and leave the other still; moving both at once is a recipe for not understanding what's going on.

The lesson I take away, after years watching how these systems behave, is almost domestic. Temperature isn't a hidden technical button for engineers: it's the tone control. When an AI sounds repetitive and bland to you, maybe it doesn't lack capacity, it has too much discipline. And when it hands you a gorgeous but unusable creative nonsense, maybe it isn't broken: it just has too long a leash for what you asked of it. Learning to move that dial is going from suffering the machine to directing it.