How ChatGPT Actually Works: A Plain English Explanation

ChatGPT can write poetry, explain quantum physics, and debug your code. It feels intelligent. But what's actually happening under the hood? The answer is more elegant—and more limited—than most people realize.

The Core Idea: Predicting the Next Word

At its heart, ChatGPT does one thing: predict the next word (technically, "token") in a sequence. Given all the text so far, what word is most likely to come next?

That's it. All the apparent intelligence emerges from doing this one thing extraordinarily well, at massive scale.

When you ask ChatGPT "What is the capital of France?", it predicts that given this sequence, the most likely continuation is "The capital of France is Paris." It doesn't "know" facts the way you do. It predicts text that would plausibly follow the input.

Step 1: Training Data

Large language models learn from text—huge amounts of it. Books, websites, articles, code, conversations. For GPT-4, we're talking about trillions of words.

During training, the model sees sequences of text and learns to predict what comes next. Shown "The cat sat on the ___", it learns that "mat" is more likely than "quantum." Across billions of examples, it learns patterns: grammar, facts, reasoning styles, coding conventions, writing tones.

The model doesn't memorize the training data. It compresses patterns into numerical weights—hundreds of billions of parameters that encode what the model has learned about how language works.

Step 2: Tokenization

Computers don't understand words. Before processing, text is converted to numbers through "tokenization."

Tokens aren't always whole words. Common words like "the" might be one token. Uncommon words get split: "tokenization" might become ["token", "ization"]. The model's vocabulary is about 50,000 tokens, and every piece of text is converted to a sequence of these token IDs.

This tokenization matters. It's why ChatGPT sometimes makes weird spelling errors or can't count letters accurately—it's not seeing individual characters, it's seeing chunks.

Step 3: The Transformer Architecture

The magic happens in the transformer architecture, introduced in the 2017 paper "Attention Is All You Need." The key innovation: attention.

Attention lets the model look at relationships between all words in the input simultaneously. When processing "The bank by the river collapsed," attention helps the model understand that "bank" refers to a riverbank, not a financial institution, by relating it to "river."

The transformer processes input through many layers (GPT-4 has ~100). Each layer refines the representation, building more abstract understanding. Early layers might capture grammar and syntax. Later layers capture meaning and reasoning.

Step 4: Generating Output

When you send a prompt, the model processes it through all those layers, then outputs a probability distribution: the likelihood of each possible next token.

It doesn't simply pick the most likely token. That would be deterministic and boring. Instead, it samples from the distribution, with a "temperature" parameter controlling randomness. Low temperature = more predictable. High temperature = more creative and chaotic.

Then it takes that generated token, appends it to the sequence, and repeats. Token by token, the response is built. When ChatGPT responds in real-time, you're watching this process: each word appearing as it's generated.

What About "Understanding"?

Does ChatGPT understand what it's saying? This is philosophically tricky. The model has no consciousness, no lived experience. It's pattern matching at superhuman scale.

But the patterns it's learned are remarkably sophisticated. It can follow logic, make analogies, and recognize context in ways that feel like understanding. Whether that constitutes "real" understanding is a question philosophers still debate.

What's certain: the model doesn't have beliefs, intentions, or desires. It's not trying to help you or deceive you. It's generating text that looks like plausible responses to your input.

RLHF: Training the Assistant

Raw language models are like talented but undirected interns. They can generate text but don't know what's helpful. This is where RLHF—Reinforcement Learning from Human Feedback—comes in.

Human trainers rate model outputs. "This response was helpful." "This one was harmful." These ratings train a reward model that predicts what humans prefer. The language model is then fine-tuned to maximize this reward.

RLHF is why ChatGPT is polite, tries to be helpful, refuses harmful requests, and admits uncertainty. These behaviors were trained in by optimizing for human approval.

The Limitations

Understanding how it works reveals the limitations:

No real-time knowledge: The model only knows what was in its training data. It doesn't browse the web or learn from your conversations.

Hallucinations: The model predicts plausible-sounding text, even when it doesn't know the answer. It can confidently generate complete nonsense that sounds authoritative.

No persistent memory: Each conversation is independent. It doesn't remember previous chats or learn about you over time (though products built on top may add this).

Reasoning limits: Complex multi-step reasoning often fails. The model can appear to reason but is really pattern-matching, which breaks down for truly novel problems.

Context windows: The model can only "see" a limited amount of text at once. Very long documents exceed this window.

Why It Works So Well

Given these limitations, why does ChatGPT feel so impressive? Scale, essentially. The patterns learned from trillions of words of human text are extraordinarily rich. Most questions you ask have been asked before, in some form. Most coding problems have solutions in public repositories. Most writing styles have been demonstrated somewhere.

The model isn't intelligent in the human sense. It's something different: an incredibly sophisticated pattern matcher that has seen enough human output to fake understanding convincingly.

Whether that's "just" statistics or something more profound is a question that gets more interesting as these models get more capable. But understanding the mechanism helps you use them better—and recognize when they're likely to fail.