How large language models work, explained in plain English

TUREFF• •0

Chatbots that write essays, answer questions and help with code are built on large language models (LLMs). You do not need maths to understand the basic idea, and understanding it helps you use them well and know when not to trust them.

Step one: learning from enormous amounts of text

An LLM starts as a huge network of numbers called parameters, set randomly. It is then trained on a vast collection of text: books, articles, websites, code and more. The training task is simple to describe: show the model some text with the next piece hidden, let it guess, and nudge its parameters so the right answer becomes a little more likely next time. Repeat this an enormous number of times. Nobody writes rules about grammar, facts or reasoning into the model. To get good at predicting the next word across so much varied text, it has to absorb patterns: how sentences are built, which facts usually appear together, how arguments and code are structured. That is why the same model can translate, summarise and write a poem.

Tokens, not words

Models do not read letters or whole words. Text is split into tokens, chunks that are often a word, part of a word or a punctuation mark. Common words may be a single token; rare words and many non-English words are split into several. When people talk about how much text a model can handle, they usually count tokens.

Generating an answer

When you type a question, the model reads your text and calculates, for every possible next token, how likely it is. It picks one (usually one of the most likely, with a little randomness), adds it to the text, and repeats, token by token, until the answer is finished. That randomness is why asking the same question twice can produce different wording. There is no database lookup behind this. The answer is produced fresh each time from patterns stored in the parameters.

Why they make things up

Because the model generates what is likely rather than retrieving what is true, it can produce confident, fluent text that is wrong. This is called hallucination. It is most common with:

specific facts the model saw rarely, such as small details, exact figures or niche topics;
references, quotations and links, which it may invent in a plausible format;
recent events that happened after its training data was collected.

Some chatbots reduce this by searching the web or reading documents you provide and basing the answer on them, but mistakes still happen. Treat important claims as leads to check, not as facts.

The context window

A model has a limited context window: the amount of text, measured in tokens, it can consider at once. That includes your messages, its replies and any documents you paste in. In a very long conversation, the earliest parts may be dropped or attended to less well, which is why a chatbot can "forget" an instruction you gave at the start. The model does not learn from your conversation in the way it learned during training; within the chat it only has what is in the window, plus any memory features the app adds.

From text predictor to helpful assistant

A model trained only to predict text is not yet a good assistant. Ask it a question and it might continue with more questions, as if completing a list. Two further steps shape it:

Fine-tuning: further training on examples of good conversations, instructions paired with helpful answers, so it learns the format of being an assistant.
Reinforcement learning from human feedback (RLHF): people compare pairs of answers and say which is better. A separate model learns to predict those preferences, and the LLM is trained to produce answers that score well. Related techniques use written principles or AI feedback in a similar way. This is how models learn to be clearer, to decline harmful requests and to admit uncertainty, though imperfectly.

Limitations worth remembering

No guaranteed truth: fluency is not accuracy.
Knowledge cut-off: without a search tool, a model knows nothing after its training data ends.
Weak at some exact tasks: counting letters, long arithmetic and precise logic can go wrong, because the model works with tokens and probabilities. Many chatbots now hand such tasks to a calculator or code tool.
Bias: models reflect patterns in their training data, including unfair ones.
Sensitivity to wording: small changes in how you ask can change the answer.
Privacy: what you type may be stored by the service. Check its settings before sharing anything personal.

Used with these limits in mind, LLMs are remarkable tools for drafting, explaining, brainstorming and coding. What would you like explained next, or what surprised you about how these models behave? Ask in the comments.