How do large language models work?
BLUF: A large language model is a neural network trained to predict the next word, or token, in a sequence of text. By learning statistical patterns from enormous amounts of writing, it generates fluent, relevant responses one token at a time.
Grasping this one mechanism explains both the striking fluency of tools like ChatGPT and their characteristic failures, such as confidently stated errors.
The prediction engine
A large language model is a type of neural network, a huge mathematical function with billions of adjustable numbers called parameters. Its one job is deceptively simple: given a stretch of text, predict what comes next. Text is first broken into tokens, which are whole words or word fragments. The model reads the tokens so far and outputs a probability for every possible next token in its vocabulary. It picks one, appends it, and repeats, building a response token by token. When you ask a question, the model is not looking up an answer in a database; it is calculating, step by step, the most probable continuation of the conversation. Fluent, on-topic writing emerges because, across trillions of training examples, plausible continuations tend to be the useful or accurate ones.
Attention and scale
The architecture behind modern models is the transformer, introduced in 2017. Its key idea is attention: when processing each token, the model weighs how much every other token in the context matters, letting it track relationships across long passages, like a pronoun and its subject or a question and its constraints. Training works by showing the model text with the next word hidden, letting it guess, then nudging its billions of parameters slightly toward the correct answer. Repeat this trillions of times and the network is forced to encode grammar, facts, reasoning patterns, and style, because all of them help predict the next word. This is why capabilities appear to emerge with scale: bigger models trained on more data compress more of the world's structure into their parameters, without anyone writing explicit rules.
Where it shows up
The same next-token engine powers a surprising range of tools. Chat assistants like ChatGPT, Claude, and Gemini are language models fine-tuned to follow instructions and refined with human feedback so their answers are helpful and safe. Coding assistants autocomplete and debug software; models draft emails, summarize documents, translate languages, and answer search queries. Because the core skill is manipulating sequences, related methods extend to protein structures and program code. Modern systems increasingly connect the model to external tools such as web search, calculators, and databases, so it can fetch current facts rather than rely only on what it memorized during training. Many now also process images and audio, not just text, by converting those inputs into the same token stream the language model was built to predict.
Common misconceptions
A common myth is that the model looks up answers in a stored database; in reality it generates each response fresh by prediction, which is why two runs can differ. People assume it always tells the truth, but it optimizes for plausible-sounding text, so it can state falsehoods confidently, a failure often called hallucination. Many believe the model understands or is conscious like a person; it has no beliefs, goals, or awareness, only statistical patterns learned from text. Finally, it is tempting to think a longer answer means the model knows more, but its knowledge is frozen at training time. Without a live tool connection it cannot know today's news, and it has no memory of past conversations unless that text is fed back into its context.