Vibe Coder · Unit 2: How Claude and ChatGPT Work · Lesson 1 of 7
What a language model does
Claude and ChatGPT do one thing underneath: predict what comes next, one token at a time. That one idea explains what they do well and badly.
Read the short notes and pictures.Do everything inside a blue frame.
One job: predict what comes next
Claude and ChatGPT are built on large language models. A language model does one thing: it reads the text so far and predicts what comes next. Writing an email, planning a product and writing code are all the same job to the model, done again and again.
Text becomes tokens
The model does not read letters or whole words. Your text is first split into small pieces called tokens. A short, common word is usually one token. A long or rare word is split into several. Each token is turned into numbers, because numbers are what the model works with.
A probability for every next token
For the text "The cat sat on the", the model gives every token it knows a probability of coming next. "mat" gets a high one. "moon" gets a low one. It picks one, adds it to the text, and predicts again. A full answer is this loop run hundreds of times.

Read
The model reads all the text so far: your message, and anything it has already written.
Predict
It gives every possible next token a probability.
Pick
It picks one. Usually a likely one, but not always the most likely, which is why the same question can get different answers.
Repeat
It adds the token to the text and starts again, until the answer is finished.
How it learned to do this
Nobody wrote rules for grammar, facts or code into the model. It was trained: shown a huge amount of text with the next token hidden, asked to guess it, and adjusted a little after each guess. After enough rounds, its guesses became good enough to write essays and working programs.

What this means for you
It is fluent
It has seen a great deal of good writing and code, so what it writes reads well and usually runs.
It writes what is likely
Likely is often right. It is not the same as true, so facts and code need checking.
It follows what you give it
Its prediction depends on the text in front of it. A clearer request gives a better answer.
It can vary
Ask twice and the wording can change, because it picks from several likely tokens.
Draft the brief, twice
Give Claude your one-line idea from "My build" and ask for a one-page product brief: the problem, the people, what the first version does. Then ask again in a new chat. Compare the two, and paste the better parts of both into "My build" under the heading "Brief".
Done when
Two drafts show you which ideas the model is sure of and which it made up on the day.
Keep these
- A language model predicts the next token, again and again.
- Text is split into tokens, and tokens become numbers.
- It learned by guessing hidden text and being adjusted, not from written rules.
- It writes what is likely, which is often but not always what is true.