You do not need to understand the mathematics of transformers to make good decisions about using ChatGPT, Claude, Gemini, or any other large language model. What you do need is a working mental model — accurate enough that you can predict where the model will help you and where it will hurt you. Here is that model, with no equations.
The Core Idea: Prediction, Not Knowledge
A large language model is, fundamentally, a very large statistical pattern recogniser. It has read an enormous quantity of text — books, articles, code, conversations — and learned what tends to come after what. When you give it some text, it predicts the next chunk of text. Then the next. Then the next. That is the entire mechanism, run at scale.
This sounds reductive, but it explains most of what LLMs do well and most of what they do badly. They produce coherent, fluent prose because they have learned what coherent prose looks like. They struggle with arithmetic and reasoning because reasoning requires more than pattern matching. They confidently invent facts because there is no internal distinction in the model between "what I learned" and "what sounds like what I learned."
The Three Stages of Training
Pretraining is the giant first stage. The model reads vast quantities of text and learns to predict the next token. After pretraining, the model is fluent but unhelpful — it will continue any text plausibly but does not know how to be useful as an assistant.
Supervised fine-tuning is the second stage. Human annotators provide examples of helpful question-answer pairs. The model learns to behave like an assistant rather than a text-completion engine. This is where the conversational behaviour comes from.
Reinforcement learning from human feedback (RLHF) is the third stage. Humans rate model outputs, and the model learns to produce outputs humans rate highly. This is where the polite, hedged, careful tone of modern assistants comes from. It is also why models tend to be over-confident on questions where they should be uncertain — they have been trained that confident-sounding answers tend to be rated higher.
A useful intuition: an LLM is like a person who has read everything ever written and remembers it all in a fuzzy, blended way. Ask it about a topic with extensive consistent coverage in the training data and it will be excellent. Ask it about something niche, recent, or contested and it will fluently make things up while sounding equally confident. The fluency is the same. The accuracy is not.
What This Mental Model Predicts
- LLMs are excellent at writing, summarising, translating, formatting, and explaining well-covered topics
- LLMs are unreliable for fresh facts, recent events, or contested claims — what they do not know, they confabulate
- LLMs are inconsistent on arithmetic and multi-step logic — pattern matching is not the same as reasoning
- LLMs reflect their training data — biases, errors, and gaps in the training data show up in outputs
- LLMs cannot reliably tell you when they do not know something — they were trained to be helpful and confident, not calibrated
- LLM outputs vary between runs — the same input can produce different outputs
How This Should Change Your Decisions
Use LLMs where fluency and pattern matching are the bottleneck — first drafts, summaries, rephrasing, generating examples, accelerating routine writing. Verify before relying on factual claims, especially anything specific, technical, or recent. Treat the output as a starting point, not a finished product. Build review into any process where the cost of an error is meaningful.
The single biggest mental shift is moving from treating LLMs as oracles to treating them as collaborators. They are not authoritative. They are productive. The user who understands that distinction gets disproportionate value. The user who does not gets confidently misled.