An interactive explainer
Type a sentence.
Watch a model think.
Everyone talks about large language models. Almost nobody has seen one work. Write anything below and we will take it apart, one stage at a time — the same four stages inside every model you have used.
Scroll
A model cannot read. It counts tokens.
Before anything else, your sentence is chopped into tokens — chunks that are usually a word, sometimes part of one, and each mapped to a number. That number is all the model ever sees. The word itself is gone.
Simplified tokeniser. Real models use byte-pair encoding over a vocabulary of roughly 100,000 pieces — dashed chips show where a word would split.
Every token becomes a point in space.
Each token id is looked up in a giant table and becomes a list of numbers — a position in a space with hundreds of dimensions. Things used in similar ways end up near each other. Nobody programmed those groupings; they fell out of predicting text. Drag to look around.
Nearest neighbours to your tokens
Illustrative layout. Real embeddings have 768 or more dimensions and cannot be drawn honestly in three — this is the shape of the idea, not the coordinates.
Each token looks at every other one.
This is attention, and it is the whole trick. For every token the model asks: which of the earlier tokens should I be paying attention to right now? Tap any token to see where it is looking.
Attention weights
Weights here are illustrative, generated by a simple heuristic. In a real model they are learned, and dozens of heads do this in parallel at every layer.
Then it guesses the next token. That is all.
The output is not a sentence. It is a probability for every token in the vocabulary. One gets picked, added to the end, and the whole process runs again from the start. Everything you have seen a model do is this loop.
The unicorn learned to write code
Candidates
A toy distribution standing in for a real model. Turn temperature to zero and it always picks the top candidate; turn it up and it takes risks — the dial behind “creative” versus “precise” modes.
Now the strange behaviour makes sense.
Why it invents citations
Nothing in that loop checks whether something is true. A fabricated reference is high-probability text — it looks exactly like a real one. Fixing this means adding retrieval and evaluation around the model, not asking it to try harder.
Why prompts change everything
Your prompt is the context every token attends to. Change it and you change every weight in stage three. That is why example order and formatting measurably move results — and why prompts belong in version control.
Why long documents cost so much
Every token attends to every other, so doubling the input roughly quadruples the work. Context windows are an engineering fight, not a setting.
Why it cannot count letters
It never saw letters. “Strawberry” may be two tokens; asking how many r’s it contains is asking about something the model was never shown.
You just read the architecture
Four stages took ten minutes.
Building one takes six months.
This is stage one of Module IV. In the full programme you build each of these yourself — a tokeniser, an embedding lookup, attention from scratch, and a model you deploy behind an API and evaluate honestly.