Mental model of LLMs: how they think and why they err

Think of an LLM as a word probability engine

An LLM does not think like a person. It predicts one tiny piece of text after another based on probabilities learned from huge datasets. Each new token is chosen because it fits the pattern of what usually follows, given your prompt and the previous text. This means the model is very good at style, structure and common solutions. It can also be confidently wrong, because it is not checking facts, it is continuing patterns. If you imagine a supercharged autocomplete that tries to be helpful and fluent, many “weird” outputs suddenly make sense.

Why surprising outputs are actually predictable

LLM behaviour feels random until you remember three forces at work. First, the prompt defines the situation and goal. Vague prompts lead to vague completions. Second, the recent conversation context shapes the answer, so earlier messages can bias or distract the model. Third, generation settings like temperature affect how adventurous the model is. When you combine these with pattern matching instead of real understanding, classic surprises appear, for example invented sources, overconfident guesses or polite but wrong summaries. With this lens, “strange” answers are usually the most probable continuation of a fuzzy or conflicting setup.

A simple mental model you can actually use

Use this everyday model: the LLM is a helpful but literal junior who finishes your sentences using internet scale experience. It does not know your intent beyond what you write. It tries to please, fills gaps with guesses and rarely admits “I do not know” unless you ask for that behaviour. Treat every answer as a first draft that follows patterns, not as final truth. When you feel surprised, ask: what pattern did my prompt suggest, which examples did I unknowingly hint at and did I signal that guessing is allowed or forbidden. This keeps your expectations grounded.

Typical thinking traps when using LLMs

Many beginners assume the model has a stable personality, a memory of past sessions or a deep understanding of their life. In reality it has none of that by default. It just uses the current conversation and patterns from training. Another trap is to ask extremely broad questions and then blame the model for generic answers. A third trap is to overlook how small wording changes can flip the most likely continuation. If you see the model as an oracle, frustration grows. If you see it as pattern completion under constraints, odd behaviour becomes a signal to adjust your prompt and context.

First steps to build intuition with low risk tests

You build a mental model by observing the model in safe practice, not on critical tasks. Start with small experiments. Ask the same question with and without clear instructions and compare the results. Change only one element at a time: role, goal, format or style. Then push on known weak spots like dates, long lists or niche facts and see how the model fails. Keep a simple note of what influences behaviour most. Over a week of short tests, you will start to predict its answers before you send the prompt, which is the real sign that your mental model works.

Useful links

• OpenAI — General information about large language models and their capabilities.
• Google DeepMind — Background reading on modern AI systems and research.

Keep learning