Guide
What is an LLM?
A large language model predicts the next token. What a token actually is, measured rather than described, and why almost every famous failure follows from it.
A large language model is a system that predicts what text should come next. That is the whole mechanism, and everything else it appears to do is that prediction applied to a prompt shaped like a request. The part worth understanding is not the prediction, which is easy to state, but the unit it predicts. Text is broken into tokens before a model sees any of it, and tokens are not words. Almost every well-known failure, from miscounting letters to being unexpectedly expensive in some languages, is that boundary showing through. The figures on this page were measured with a real tokeniser rather than taken from anybody's summary.
Last reviewed:
The whole mechanism, in one sentence
A large language model predicts what text should come next.
It was trained on an enormous quantity of text until it became very good at that single task. Answering a question, writing code, summarising a document: all of it is the same prediction, applied to a prompt shaped like a request. There is no separate reasoning module that switches on for hard questions.
That sentence is easy to accept and easy to underestimate. The interesting part is not the predicting. It is the unit being predicted, because the model does not work in words and never has.
Tokens, measured rather than described
Before a model sees your text, a tokeniser cuts it into fragments. Common sequences become one token. Rarer ones are split. The vocabulary is fixed when the tokeniser is trained, and it is chosen by frequency in the training data rather than by any rule about meaning.
Here is what that actually does, run through Gemini's tokeniser rather than paraphrased from anybody's blog post.
Five strings, and where the tokeniser cuts them
strawberry un | bel | ievable hall | uc | ination CB | 2 | | 1 | TN 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 0
- 01
strawberry: One token. A single indivisible symbol with no internal structure the model can look at. - 02
un | bel | ievable: Three tokens, and not where a linguist would divide it. The split is statistical, not morphological: nothing here knows that un is a prefix. - 03
hall | uc | ination: Three tokens. A common word in AI writing that the tokeniser still does not hold whole. - 04
CB | 2 | | 1 | TN: Five tokens for a UK postcode. Identifiers fragment badly, which is why exact-match lookups on codes and part numbers are unreliable. - 05
1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 0: Ten tokens, one per digit. The model has no representation of this as a number at all, which is a large part of why arithmetic is unreliable.
Two of those rows explain failures that get reported as though they were mysteries.
Why nobody can count the letters in strawberry
The best-known embarrassment in this field is a model insisting that strawberry contains two letter r. It is treated as evidence that these systems are stupid. It is better understood as evidence that they do not see what you see.
Strawberry is one token. The model receives a single symbol. It has no access to the sequence of characters inside it, in the same way that you have no access to the individual pen strokes in a word somebody speaks aloud. Asking it to count letters is asking it to inspect something that was discarded before it was handed the input.
Models often answer correctly now, and the reason matters: they have read a great deal of text discussing how strawberry is spelled. That is recall about the word, not inspection of it. The distinction shows up the moment you pick a word nobody has written a blog post about.
- 1Training data Text it was shown once, months ago
- 2Parameters Numbers fixed at release, not a memory
- 3A distribution Every possible next token, ranked
1
Token in strawberry
No internal structure. The letters are not available to be counted.
10
Tokens in a ten-digit number
One per digit. Nothing represents it as a quantity.
3.4x
More tokens for Japanese
Per character, against English. The same content costs more to process.
The same explanation covers arithmetic. A ten-digit number arrives as ten separate symbols with no numeric meaning attached to any of them. That a model can do arithmetic at all is the surprise. That it does so unreliably is what the representation predicts.
The practical rule that falls out of this: anything depending on the exact characters of a string, counting them, reversing them, matching an identifier precisely, belongs in code rather than in a prompt. Not because the model is weak, but because it is working from a representation that threw those characters away.
The language tax nobody quotes you
Tokenisers are trained overwhelmingly on English text, so English gets the efficient vocabulary. Other languages are cut into smaller pieces. This is usually described in the abstract; it is straightforward to measure.
Tokens per 100 characters of text
Japanese costs roughly 3.4 times as many tokens per character as plain English. Arabic and Hindi land near double. That has three consequences that reach a budget rather than a research paper.
Cost. Billing is per token. The same document, professionally translated, costs several times more to process in some languages than in others, for identical meaning.
Context capacity. A context window measured in tokens holds proportionally less Japanese text. A limit that comfortably fits a long English contract may not fit its Japanese equivalent.
Latency. More tokens is more compute, both to read the prompt and to generate the reply.
None of this is announced. It falls out of a design decision made when the tokeniser was trained, and it is invisible until somebody compares two invoices.
What happens between your prompt and the answer
One pass, repeated
- Text to fragmentsTokenise
- Fragments to vectorsEmbed
- Weigh the contextAttention layers
- Over the vocabularyProbabilities
- Append and repeatSample one
Each token is turned into a vector, a long list of numbers positioning it in a space where related meanings sit near each other. Those vectors pass through many layers of attention, a mechanism that lets each position weigh every other position: this is where the model works out that a pronoun refers to a noun forty words earlier.
The output is a probability across the entire vocabulary for what comes next. One token is selected, appended to the input, and the whole sequence runs again from the beginning. A three-hundred-token answer is three hundred complete passes.
Parameters are the numbers inside those layers, fixed at the end of training. When a model is described as having some number of billions of parameters, that is the count of adjustable values that were tuned. They do not change while you use it. Nothing you type alters the model.
Temperature decides how the selection is made. At zero the most probable token is always taken and the output is close to deterministic. Higher values sample further down the distribution, which is what produces variety and also what produces confident nonsense.
It is not only text any more
Describing these systems as language models is now slightly behind the product. Current frontier models accept images, audio and video alongside text, and the mechanism does not change to accommodate them: those inputs are converted into tokens too, and then predicted over in exactly the same loop. An image becomes a few hundred tokens; a minute of audio becomes several hundred more.
Two things follow that catch people out.
Images consume the context window like everything else. A conversation with a dozen screenshots in it has spent a substantial part of its budget on them, and the earlier text starts falling out of reach sooner than expected.
The security surface widens with every modality. If a model reads an image, then text inside that image is input. An instruction printed in a screenshot, or set in pale grey on a white background where a human eye slides past it, arrives in the context with the same standing as anything else. The channel is new; the underlying problem is the one already described, that instructions and content are indistinguishable once they are tokens.
For anybody assessing one of these systems, the practical question is simply which modalities are enabled, because each one is another route by which untrusted content reaches the model.
The context window, and why bigger is not simply better
The context window is the total number of tokens a model can consider at once, counting your prompt, the conversation so far, any documents you pasted, and the reply being generated. Everything competes for the same budget.
Vendors compete on this number and it is easy to read as a capacity spec: larger window, more room, problem solved. The measured behaviour is less convenient.
Liu and colleagues, in Lost in the Middle (TACL, 2023), placed the same relevant passage at different positions inside a long context and measured whether the model could use it. Accuracy was highest when the passage sat near the beginning or the end, and fell noticeably when it sat in the middle. The information was present every time. Whether the model used it depended on where it was.
Where in the context a fact sits changes whether it is used
Three practical consequences, none of which a larger window fixes.
Filling the window is not the same as being read. Pasting an entire policy manual and asking a question about clause 40 puts clause 40 exactly where attention is weakest.
Position is a design decision. If something must not be missed, put it at the start or the end of the prompt, not buried in the middle of supporting material.
Retrieval is partly an attention optimisation, not only a cost one. Selecting eight relevant passages and placing them near the question puts the material where the model reliably reads, which is a separate benefit from the saving on tokens.
What "large" actually refers to, and the choice it implies
Large describes two things: the parameter count, and the volume of training text. Neither is a quality score, and the industry has quietly stopped competing on parameter count because smaller models trained better now beat larger models trained worse.
The distinction that matters more for anybody procuring one is where the weights live.
Hosted models are called over an API. You get the strongest available capability with no infrastructure, and in exchange your text leaves your estate, the model can change under you without notice, and your costs scale with use forever. A prompt validated in March can behave differently in September because the vendor updated something.
Open-weight models are downloaded and run on your own hardware. The weights are fixed, so behaviour is stable and reproducible, and nothing leaves. In exchange you own the serving, the hardware, the scaling and the upgrades, and the frontier capability is generally somewhere ahead of you.
For regulated work the deciding factor is usually neither capability nor cost. It is that an open-weight model can be pinned to a version and evidenced, which is a much easier conversation with an auditor than explaining that the system you validated has since been silently replaced.
How a model gets made
Understanding the three stages explains why a model behaves the way it does, and which behaviours a vendor can change without retraining anything.
From raw text to the thing you talk to
- 1
Pretraining
Months, enormous cost
Predict the next token across a very large corpus. This is where capability and factual recall come from, and it is the only stage most organisations will never do.
- 2
Supervised fine-tuning
Days to weeks
Train on curated examples of instructions and good responses, which is what turns a text predictor into something that answers a question rather than continuing it.
- 3
Preference alignment
Ongoing
Humans rank competing responses and the model is tuned toward the preferred ones. This produces tone, refusals and safety behaviour, and it is the stage most often adjusted after release.
- 4
System prompt and tools
Instant, per product
Not training at all: instructions prepended at request time, plus any tools the product exposes. Most of what distinguishes two products built on the same model lives here.
That last row is worth internalising when assessing a vendor. A great deal of what is sold as a proprietary model is a system prompt and a tool list wrapped around somebody else's weights. That is not necessarily bad, and it is a different risk profile, a different cost base and a different answer to who sees your data.
What follows from the mechanism
Every one of these is a consequence of what has already been described, not a separate flaw to be fixed in a later version.
What it does well
- Fluent text in a requested style, at length.
- Transformation of text you supply: summarise, rewrite, translate, extract.
- Pattern completion over familiar structures, including code.
- Answering from material placed in the prompt, which is why retrieval works.
What the mechanism prevents
- Knowing whether its own output is true.
- Anything requiring the characters inside a token, such as counting or exact matching.
- Reliable arithmetic, for the same representational reason.
- Remembering anything between calls without a system built around it.
Hallucination is structural. The model selects likely tokens. Plausible and true are different properties, and only the first is being optimised. A fluent, well-formatted, entirely wrong answer costs it exactly as much effort as a correct one, which is why it produces them with the same confidence.
There is no memory. Every request is stateless. What looks like a conversation is the application resending the whole exchange each time, which is why costs climb through a long chat and why instructions given at the start begin to lose against everything piled on top of them.
Instructions and content arrive as the same kind of text. The model has no channel that distinguishes what you told it from what arrived in a document it was asked to read. That is the whole basis of prompt injection, and it is why the problem is architectural rather than a bug awaiting a patch.
Choosing one, and the questions that separate them
Model comparisons are usually conducted in benchmark scores, and benchmark scores are close to useless for deciding what to buy. They are public, so they are optimised for; they measure tasks that are not yours; and the differences between the top few models are routinely smaller than the variation you get from rewriting a prompt.
The questions below separate real options, and none of them is about capability.
What happens to our text? Retained or not, for how long, reviewed by humans or not, used for training or not. This is a contractual answer, not a technical one, and it is the answer that decides whether a model can touch regulated data at all.
Can we pin a version? If the vendor can change the model underneath a validated workflow, then your validation has an expiry date nobody told you about. Ask what the deprecation notice period is and what happens to the old version at the end of it.
What does it cost at our actual volume, in our actual languages? Take a representative document, count its tokens, multiply. The language measurements above exist because that arithmetic surprises people who assumed a per-seat price.
Where does it run? Which jurisdiction, under whose law, and does that survive a data protection assessment. For a UK or EU organisation this is frequently the shortest route to eliminating half the shortlist.
How does it fail? Ask for the refusal behaviour, the rate limits, and what happens on overload. A model that returns a 429 under load needs a different application design from one that queues.
Then measure the shortlist on your own work rather than on anybody's leaderboard. Twenty representative tasks, a defined right answer for each, three runs apiece to expose variability. That is an afternoon and it produces a number that means something, which no published benchmark will do for you.
How these figures were produced
Every number here came from the local tokeniser in Google's genai SDK, model gemini-2.5-flash, run in August 2026 against short strings of comparable content. Tokens per 100 characters is used for the language comparison because word counts are not comparable across writing systems: Japanese does not separate words with spaces, so tokens per word would flatter it enormously and mean nothing.
Other tokenisers differ. The direction of every finding here is a property of how tokenisers are built rather than of one vendor, but the exact figures belong to this one, and anybody quoting them should say so.
Where to go next
The explainer on this site runs the loop in front of you: tokenise, predict, sample, repeat, with a temperature control that visibly changes the output. It is the fastest way to stop thinking of a model as something that knows answers.
If the security consequence is the pressing part, the pattern library catalogues the forms an injected instruction takes, with the text of each.
If the question is how to give a model your own documents without retraining anything, that is retrieval, and it has its own guide.
Common questions
›What does LLM stand for?
Large language model. Large refers to the parameter count and the volume of training text; language model is the technical term for a system that assigns probabilities to sequences of text. The name is unusually literal for the field.
›How does an LLM actually work?
Your text is split into tokens, which are fragments rather than words. Those tokens pass through the network once and produce a probability distribution over the whole vocabulary for what should come next. One token is chosen, appended to the input, and the entire process runs again from the start. Fluent paragraphs are that loop repeated a few hundred times.
›What is a token in an LLM?
A fragment of text, sized by how often it appeared during training. Common words are one token; rarer words are split. Measured with Gemini's tokeniser, unbelievable becomes three tokens, un plus bel plus ievable, which is not where a linguist would divide it. Tokens are the unit of billing and the unit of the context limit, so they are worth understanding before either surprises you.
›Why can't ChatGPT count the letters in strawberry?
Because it never sees the letters. Measured with Gemini's tokeniser, strawberry is a single token: one indivisible symbol with no internal structure the model can inspect. Asking how many times r appears in it is like asking someone to count the strokes in a word they only ever heard spoken. The model can often answer correctly by recalling text about the spelling, which is a different skill from looking.
›Do LLMs cost more in some languages?
Yes, and measurably. The same tokeniser turns 100 characters of English into about 23 tokens and 100 characters of Japanese into about 77, so identical content costs roughly three times as much to process in Japanese. It also consumes the context window three times faster. Tokenisers are trained on text that is predominantly English, and the price difference follows from that.
›Why do LLMs hallucinate?
Because plausible and true are different properties and the model optimises for the first. It selects tokens that are statistically likely given the context, and a confident, well-formed, entirely incorrect answer costs it no more effort than a correct one. Hallucination is not a defect bolted onto an otherwise truthful system. It is the same mechanism working exactly as designed.
›Does an LLM remember previous conversations?
No. Each request is stateless. What looks like memory is the application resending the earlier conversation as part of the new prompt, which is why long chats start losing instructions given near the beginning and why every turn costs more than the last. Any memory beyond that is a database somebody built around the model.
›What is the difference between an LLM and AI?
An LLM is one kind of AI. Artificial intelligence is the broad category of systems that infer rather than follow written rules; machine learning is the subset that derives behaviour from data; deep learning is the subset using many-layered neural networks; and large language models are deep learning applied to text.
Where to go next