Library

AutoTemp

AutoTemp: The tool that tunes an AI’s creativity by testing it

Every AI model has a dial that sets how predictable or how adventurous it is. This tool tries several settings at once and keeps the best answer.

What this means for you

When an AI model writes, it chooses one word at a time. At each step it has a ranked list of words that could plausibly come next. Whether it takes the top choice every time, or sometimes reaches further down the list, is controlled by a single setting. The setting is called temperature.

At a low temperature the model plays it safe. It picks the most likely word, again and again. The result is steady, consistent and a little flat. At a high temperature it takes more chances. The result can be fresher and more surprising, and it can also wander off or stop making sense. Ask the same question twice at a high setting and you may get two quite different answers.

The DialΛΥΣΙΟΣ · LIBERHow likely each next word is, at three settingsprimumprimumprimumICOLDThe likeliest word, every timeIIWARMMostly likely, sometimes notIIIHOTAlmost any word has a chance
Fig. ITemperature changes the odds. Cold, one word takes nearly all of them. Hot, the odds are spread almost evenly across many words.

Most people never see this dial. The chat apps set it for you and do not mention it. People who build things with AI do see it, and they face a small, constant annoyance: the right setting depends on the job. A legal summary wants a cool, careful temperature. A list of names for a new bakery wants a warmer one. There is no single correct number, so in practice people guess, look at the result, and guess again.

It is like cooking a steak when you do not know how your guest likes it. You could guess. Or you could cook several small pieces, one rare, one medium, one well done, let a few people taste them, and serve whichever one they liked best.

AutoTemp does the second thing. You give it one request. It sends that request to the model several times, each at a different temperature, all at once. Then it asks the AI to act as a panel of judges, scoring every answer on things like how relevant it is, how clear, how useful and how creative. The answer with the best score is the one you get.

The TastingΛΥΣΙΟΣ · LIBER?Libros Naturalis Historiaenovicium Camenis Quiritiumtuorum opus natos apud meproxima feturaVIVVIconstitui tibi iucundissimeImperator sit enim haec tuipraefatio verissima dummaximi consenescitIXVIIIIXnugas esse aliquid measputare res ardua vetustisnovitatem dare novisauctoritatem obsoletisVIIVVIOptimumconstitui tibi iucundissimeImperator sit enim haec tuipraefatio verissima dummaximi consenescit in patrenugas esse aliquid measputare res ardua vetustisnovitatem dare novisauctoritatem obsoletisIONE REQUESTAsked onceIITHREE DRAFTSCold, warm and hotIIITHE JUDGESEach draft scored three timesIVBEST DRAFTThe one you get back
Fig. IIWhat AutoTemp does with one request: write it at several temperatures, have a panel score every draft, and hand back the winner.

Why a panel of judges

A single judge has moods. Ask one AI judge to score an answer and the score can swing for no good reason. AutoTemp runs several independent judges and averages them, the same way a diving competition uses a row of scorecards. It is a small design choice that makes the result steadier.

There is also a smarter mode for when you want to spend fewer tries. Instead of testing every temperature equally, it starts by sampling each one, notices which are doing well, and gives those more attempts while still checking the others occasionally in case it was wrong about them. It is the strategy a sensible person uses with a row of slot machines, and it has a name, which is covered on the technical side.

The wider point is modest and worth having. An AI’s answer is one draw from many possible answers. A setting you never see shapes which draw you get. Knowing the dial exists makes you a better reader of anything a model tells you: if an answer matters, ask again.

How to use it

AutoTemp is free on GitHub at the link above, and you can read how it works without an account. To run it you need two things that put it a step beyond casual use. One is a little comfort with installing software. The other is a paid key from OpenAI, which is what lets the tool send requests to the model on your behalf. Each test uses that key several times over, once per temperature and again for the judges, so it costs more than a single question would.

If you have both, there are two ways in. The project has a simple web page version at elder-plinius.github.io/AutoTemp: you paste in your key, type a request, list the temperatures you want to compare, and press run. The key stays in your own browser. Or you can download it and run it on your computer, which opens the same kind of form. Either way you see every answer side by side with its scores, which is the instructive part.