How a model picks the next word.
A language model does not retrieve a sentence from a drawer. It scores possible next tokens, turns those scores into probabilities, and samples one. This lab lets you inspect that distribution and turn the randomness up or down.
Interactive lab
Sample the next token
Lower temperature concentrates probability on likely tokens. Higher temperature gives unlikely tokens more room.
themodel…
Each token is drawn from the probability bars on the right.
after “model”learns50%makes25%predicts25%
At temperature 1.00, these bars match the counts in the training sentences. The model has seen 128 tokens and 48 distinct tokens.
Read the training sentences
This is the complete training data. A real language model learns from a vastly larger corpus and a much wider context than one word.
the cat sleeps on the warm mat .the cat watches a small bird .the cat chases a small ball .the dog sleeps on the warm rug .the dog chases a small ball .the dog watches the quiet street .the sun rises over the quiet town .the sun sets behind the quiet hills .a model learns from many examples .a model predicts the next token .a model makes a careful guess .the model learns from the examples .the student asks a careful question .the student tests a simple idea .the student learns from the mistake .a guess can be wrong .a guess can be useful .the next token depends on context .
01 — The unit
A model writes one token at a time.
A token is a chunk of text. In this lesson each token is a word or a punctuation mark, but production tokenizers often split words into smaller pieces. The model's job is always the same: given the tokens so far, produce a score for every token that could come next.
Those scores are called logits. They are not probabilities yet: they can be negative, and they do not sum to one. A softmax turns them into a probability distribution. Then the system samples a token from that distribution, appends it, and starts again.
A whole response is this loop repeated. The model does not plan the entire sentence before writing it; each new token changes the context for the next decision.
02 — The distribution
This model counts what comes next.
The lab uses a bigram model, the smallest useful language model. During training it reads the corpus and counts which token follows each token. Given the prompt the cat, it looks at the final token, cat, and retrieves the continuations it saw: sleeps, watches, and chases.
At temperature 1.00, a token seen twice as often gets twice the probability. With three equally frequent continuations after cat, each gets 33%. The bars in the lab are the actual counts after softmax, not a mocked illustration.
z(token) = ln(count)
p(token) = ez / T / Σ ez / T
T is temperature. At T = 1, the softmax of log-counts reduces to count ÷ total.
Real language models learn their logits with neural networks rather than direct counts. The softmax and the sampling step at the end are the same shape.
03 — The dial
Temperature trades surprise for reliability.
Low temperature sharpens the distribution. A token with a small lead becomes a much safer bet, so the model sounds focused and repetitive. High temperature flattens the distribution. Less likely tokens get a real chance, so the output becomes more varied and more likely to go somewhere odd.
Temperature does not add knowledge. It only changes how the existing probabilities are sampled. If the model has never seen a token in a given context, raising the temperature cannot invent that context.
At the extreme low end, sampling approaches greedy decoding: always take the single most likely token. At the extreme high end, the distribution approaches uniform, which is rarely what you want from a language model.
04 — Your turn
Four experiments to try.
Make “the” choose
Start with the model and look at the bars. At T = 1.0, learns gets 50% while makes and predicts get 25% each.
Now drop the temperature to 0.2. learns takes almost all the probability; the other continuations fade to a few percent.
Add some chaos
Keep the prompt and raise the temperature to 2.0. Sample a few times.
The bars flatten and unlikely continuations appear more often — including paths that surprise you.
Follow one path
Prompt the model, set T = 0.6, and sample the same prompt three times.
The outputs differ because sampling is random. The probability distribution is fixed; the draw is not.
Break the context
Prompt with a word outside the corpus, such as volcano. The lab reports that it has no examples after that token.
A model can only continue patterns it has learned. Here the training data is visible, so you can see exactly why it fails.
05 — Scope
What this tiny model leaves out.
The probabilities, softmax, and sampling are real. The language modeling is deliberately minimal.
- One token of context. The model looks only at the final token, not the whole prompt. A transformer attends across many tokens at once.
- Eighteen sentences. The corpus is readable on purpose. A production model trains on a vastly larger and more varied body of text.
- Counts, not learned weights. There are no embeddings, attention layers, or backpropagation here. The transition table is the model.
- Word-level tokens. Real tokenizers split text into subword pieces so they can represent rare words and punctuation efficiently.
Keep this
A language model produces a probability distribution over the next token. Softmax builds the distribution; temperature reshapes it; sampling picks one outcome. Repeat that loop and you have generated text.