A language model is autocomplete that read the whole internet

The second stop on my AI roadmap is the one that stops a model feeling like magic. You do not need the maths to build with one, but you do need a working picture of what happens between your question and its answer, because every strange behaviour you will hit later, the made-up facts, the forgotten context, the sudden cost, comes straight from that picture.

It is guessing the next word, very well

Here is the whole idea in one line: a language model (LLM, large language model) is a program that predicts the most likely next chunk of text, given everything so far. That is it. Ask it a question and it does not look the answer up. It generates the answer one piece at a time, each piece being its best guess at what should come next, based on patterns from an enormous amount of text it was trained on.

The autocomplete on your phone does this with three words of history. A language model does it with billions of parameters and, in effect, a large chunk of the written internet as its sense of “what usually comes next”. That scale is why the guesses are good enough to feel like thinking.

flowchart LR
  P[Your prompt] --> T[Split into tokens]
  T --> M[Model predicts the next token]
  M --> A[Append it]
  A -->|repeat until done| M
  A --> R[The answer]

Those chunks are called tokens: a word, part of a word, or a piece of punctuation. Roughly, a thousand tokens is about 750 English words. Tokens matter to you for one practical reason: you are billed per token, in and out, and everything about the model’s memory is measured in them.

The memory limit that explains the forgetting

A model has no memory between calls. What it can “see” at once is the context window: your instructions, the conversation so far, any document you pasted, and the answer it is writing, all have to fit inside it, measured in tokens.

This explains a behaviour that confuses everyone at first. A long chat starts to forget things you said earlier, not because the model is careless, but because the oldest material fell out of the window. Bigger windows help, but the window is always finite, and managing what goes in it is a real skill you will meet again at later stops.

The dials: temperature and top-p

Two settings shape how a model picks each next token, and you will see them in every API.

Temperature controls how adventurous the guess is. At 0, the model always takes the single most likely token, so the same prompt gives nearly the same answer every time. Higher values let it pick less likely tokens, which reads as more varied or creative, until somewhere above 1.3 it becomes incoherent. Top-p is a related dial that limits the pool of tokens it may choose from.

Why it confidently makes things up

The design above has a consequence that is the single most important thing to understand before you ship anything: the model is trained to produce plausible text, not true text. When it does not know, it does not stop. It keeps predicting the most likely-sounding continuation, and the result is a hallucination, a confident, fluent, wrong answer.

Two model-level causes to know by name. The knowledge cutoff is the date its training data ends: it knows nothing after that, so ask about last month and it may invent it. And gaps in its training simply get filled with the most probable-sounding thing.

Which model, and open or closed

Models come in sizes, and picking one is a lot like picking a car class. A small, fast model is a city car: cheap, quick, fine for simple trips like classifying text or short answers. A large model is the van: slower and dearer, but it handles heavy reasoning and long, messy tasks. Most real apps use both, routing easy work to the small one.

The other split is closed versus open. Closed models (from Anthropic, OpenAI, Google) you reach through an API and pay per token; they are generally the strongest and you never touch the weights, the numbers inside the model that training produced. Open models (Llama, Mistral, and others) you can download and run yourself, which buys privacy and control at the cost of running the hardware. Many models are now multimodal, meaning they take images or audio as input, not only text.

The tools you actually reach for

Autocomplete, but treat it like an intern

The mental model to carry forward is unglamorous and very useful. A language model is a spectacular autocomplete: it predicts, it does not know; it has a memory window, not a memory; it has dials you can turn; and it fills gaps with confident guesses. None of that makes it less powerful. It just tells you exactly what your job is around it: give it the right context, set the right dial, and never ship what it says unchecked.

It predicts. You verify.

Sources

• Temperature, tokens, and context windows: the three pillars of LLM control for tokens, the context window, and the temperature dial
• MLQ: tokens and context windows in LLMs, explained for how token counts and the window work
• Atlan: LLM hallucinations, why they happen and how to reduce them (2026) for hallucination as a consequence of next-token prediction
• Wikipedia: knowledge cutoff for what the training cutoff means