A model never sees words
It sees tokens — chunks of bytes with numbers attached. Everything odd about how models spell, count, price and forget starts here.
4 min · Playable
Type something. Watch what it actually costs.
Every tile is one token. Colour tells you what kind it is, the number underneath is its id, and the meters on the right turn that into money and context. Try the samples — the number and the emoji are the ones that surprise people.
Cost to send this, one million times
$36.00$3.00 per million input tokens
Context window used
12 / 128,000 · 0.009%
An approximation of a GPT-style byte-pair tokeniser, not the real vocabulary. Counts land within about 10% of the real thing for English and are rougher for other scripts — the behaviour it demonstrates is exact.
Think you can guess the count?
Seven rounds. Ten points for an exact hit, fewer the further out you are. Nearly everybody scores badly on the emoji and the Hindi, which is precisely the point.
The cat sat on the mat.
Short, common words. Easy mode.
The strawberry problem, settled
The famous failure is not a reasoning failure. Put your own word in and watch the letters disappear.
You see letters
Counting is trivial. There are 3 of them, and you can point at each one.
The model sees pieces
3 opaque ids. The letters are gone before the model starts — so it has to infer the answer from text it has read about spelling, rather than by looking.
The same sentence costs more in Hindi
Tokenisers are trained on text that is mostly English, so English gets the efficient pieces. Everyone else pays a tax — in money, in latency, and in how much fits in the window.
The baseline. Trained on mostly this.
Close cousin, mild penalty.
Devanagari costs several tokens per word.
Roughly one token per character.
Each glyph is several tokens of bytes.
The same meaning costs several times more in some languages than in English — in money, in latency, and in how much fits in the context window. That is a fairness problem, not a technical footnote.
Four mysteries, one cause.
Why it cannot count letters
The letters are gone before the model starts. Asking how many r’s are in “strawberry” is asking about something it never saw — it can only recall what it has read about spelling.
Why your bill is not proportional to words
You are charged per token, and tokens are not words. Code, numbers, emoji and non-English text all cost far more per unit of meaning than plain English prose does.
Why context windows fill faster than expected
A 128k window is not 128k words. For a codebase or a Hindi corpus it might be a fifth of what you assumed, which is usually discovered in production.
Why rare words behave oddly
A name split into five unfamiliar pieces has no single representation to draw on. This is why models misspell unusual names and mangle rare technical terms.
Next
Stage one of four. Want to see the rest?
Tokens are the first of four stages inside a language model. The next lab takes apart all of them, with your own sentence.