LLM Temperature Explained – LLM Tutorial

By | March 21, 2024

When we use LLM model to generate a text, we may see a temerature parameter. In this tutorial, we will discuss it.

Look at code below:

temperature in llm

The effect of temperature in LLM

When llm generate a output, it may use a softmax function get the probability of the next token. It can be defined as:

The effect of temerature in LLM

Here T is the temperature. It is:

\(T\gt 0\)

To understand the effect of T, we can read this tutorial:

An Explanation of Softmax Function with Hyperparameter – Machine Learning Tutorial


\(T = \frac{1}{S}\)

As to T:

0<T<=1: It will be benefit to big probability, which will decrease the llm output versatility.

T>1: It is benefit to small probability, which will make llm generate more creative outputs.

Here is an illustration.

The effect of different temperature