AI Skills Academy · Unit 2 of 8
Unit 2: How chatbots work
Next-word prediction, tokens, training, randomness and context: what is really happening when a chatbot answers.
Opens the first lesson of this unit in AI Skills Academy. Every lesson is open from the start.
Lesson 1: Predicting the next word
Goal: Explain next-word prediction and what a token is.
The chatbots you can talk to are built on large language models (LLMs). At their core, they do one thing over and over: given some text, predict what comes next.
Give a model "The cat sat on the" and it scores possible continuations: "mat" very likely, "floor" likely, "volcano" very unlikely. It picks one, adds it to the text, and predicts again. A whole answer is built this way, one piece at a time.
Models do not actually work in whole words. They split text into tokens: common words are often one token, and rarer words are split into pieces. A rough rule of thumb for English is that a token averages about three-quarters of a word.
Hands-on lab in the room: Next-word predictor. This tiny model learned from a few short paragraphs. Type a starting word to see which words it thinks come next, then let it write.
Here is the surprising part. To predict text well across billions of examples, a model has to pick up grammar, facts, writing styles and even patterns of reasoning, because all of those help predict what comes next. That is why prediction can look like understanding. It is also why a model can produce smooth, confident text that is simply false: fluent and plausible is what it is built to produce.
- Large language model (LLM)
- A very large model trained on huge amounts of text to predict the next token. Chatbots are built on LLMs.
- Token
- A chunk of text a model reads and writes: a word or part of a word.
Remember: A chatbot writes one token at a time by predicting what plausibly comes next. Plausible is not the same as true.
Lesson 2: Training: from raw text to helpful assistant
Goal: Describe pretraining and fine-tuning, and what each one means for you.
Turning a pile of text into a helpful assistant happens in stages.
Pretraining. The model reads a huge amount of text (books, websites, articles, code) and practices predicting the next token, billions of times. Each time it is wrong, its internal numbers, called weights or parameters, are nudged so it does a little better. This is where it picks up language and a lot of general knowledge, along with the mistakes, gaps and biases in that text.
Fine-tuning. A pretrained model is good at continuing text but not at being a helpful assistant. So it is trained further on example conversations, and people compare and rate its answers. The model is then trained toward the kinds of answers people rated as more helpful, honest and safe. One common method is called reinforcement learning from human feedback (RLHF).
Worked example: why this matters to you
- The model does not keep a searchable copy of everything it read. What it learned is spread across billions of weights, a bit like how you remember the gist of a book but not every page.
- So it can mix up details, like two people with similar names, or an author with the wrong book.
- And because people rewarded helpful-sounding answers, it may try to give an answer even when "I am not sure" would be more honest.
Treat specific facts from a chatbot as leads to check, not as finished research.
- Pretraining
- The first, biggest stage: learning to predict the next token across a huge amount of text.
- Fine-tuning
- Further training on targeted examples so the model behaves a certain way, such as following instructions.
- Weights (parameters)
- The model's internal numbers, adjusted during training. They store what it learned.
Remember: Pretraining teaches language and knowledge. Fine-tuning teaches behavior. Neither gives the model a perfect memory.
Lesson 3: Why answers change: randomness and temperature
Goal: Explain why the same question can get different answers, and use that as a check.
Ask the same question twice and you may get two different answers. That is on purpose. Instead of always taking the single most likely next token, the model usually samples: it picks from the likely options with some randomness, like rolling weighted dice.
Many AI tools have a setting called temperature that controls how much randomness there is.
- Low temperature: sticks to the most likely tokens. Answers are more predictable and repeatable: good for facts, steps and code.
- High temperature: takes more unusual options. Answers are more varied and creative, and also more likely to go off track: good for brainstorming and stories.
Hands-on lab in the room: Next-word predictor. Slide the temperature and press Write a few times. Watch how low values repeat themselves and high values get strange.
You can use this as a test. Ask a factual question twice, or in a fresh chat. If the answers disagree on the facts, that is a strong warning that the model is guessing. For creative work, regenerating is a free way to get more options.
- Sampling
- Choosing the next token at random from the likely options, weighted by how likely each one is.
- Temperature
- A setting for how much randomness is used when sampling. Lower is more predictable, higher is more varied.
Remember: Different answers to the same factual question are a red flag. Different answers to a creative request are a free source of options.
Lesson 4: Context: what the chatbot can see
Goal: Use the context window on purpose and know what a chatbot does and does not remember.
Every time a chatbot replies, it reads everything in front of it right now: the app's hidden instructions, your messages, its own earlier replies and any files you shared. All of that together is the context. The maximum it can take in at once is its context window, measured in tokens.
Two things follow from this. First, the model can only use what is in its context or what it learned in training. If you know a key fact, put it in the prompt. Second, very long chats can push early details out of view or blur them, so the model starts drifting or forgetting what you agreed on.
Does a chatbot remember you? The model itself does not learn from your chat while you talk. Some apps add a memory feature that saves notes about you and adds them to the context in later chats. Check your app's settings so you know which kind you are using.
Worked example: keeping a long project on track
- You have been planning a party in one chat for an hour, and the bot starts suggesting things you already ruled out.
- Ask it: "Summarize everything we have decided so far as a short list."
- Check and fix the list, then start a fresh chat and paste the summary first.
A short, correct summary at the top of a fresh chat keeps the important facts in view.
- Context
- Everything the model can read when it writes its reply: instructions, the conversation so far and any shared files.
- Context window
- The maximum amount of text, in tokens, the model can take in at once.
Remember: If it matters, put it in the prompt. If the chat gets long and messy, summarize and start fresh.
Practise it in AI Skills Academy
Each lesson ends with two or three questions that check you can use the idea, not just recognise it. Wrong answers explain why and come back in Review. The unit ends with a 6-question quiz; pass it with 80% or more to master the unit. Hands-on lab: Next-word predictor.
Every lesson is free: no ads, no lives and no paywall.