We have all the parts — attention, Q/K/V, positional order. Now let's assemble the full Transformer and meet its two families: the Encoder (the "Understander") and the Decoder (the "Writer"). This is where we finally name GPT and BERT.
The blueprint
The original Transformer: two halves
The 2017 paper laid out an architecture with two parts working together: an encoder and a decoder. The encoder understands the input; the decoder writes the output. This pairing is perfect for tasks like translation or answering questions — understand one language, produce another.
Encoder — the "Understander"
Reads & absorbs meaning
Looks at every word at once (self-attention, both directions)
Realizes "sits" refers to "cat", "on" refers to "mat"
Outputs a context-enriched summary of the whole sentence's meaning
e.g. "The cat sits on the mat" → its full meaning
Decoder — the "Writer"
Writes output, word by word
Uses self-attention on what it's already written (so it doesn't repeat)
Uses encoder-decoder attention to check the original meaning
Produces the output one word at a time
e.g. → "Le chat est assis sur le tapis"
Layman example — a translator duo
Imagine translating English → French with two people. The Reader (encoder) reads the whole English sentence and deeply understands it — the gist, the nuance. Then the Writer (decoder) takes that understanding and writes it out in French, one word at a time, glancing back at both what they've written and the original meaning. Two specialists, one pipeline.
🎮 The blueprint — interactive
The full Transformer architecture (click any block)
This is the actual diagram from the 2017 paper, redrawn. Data flows bottom → top. The encoder stack is on the left, the decoder stack on the right. Click any block to learn what it does.
The Transformer — encoder (left) + decoder (right)
Bottom = input · Top = output. Click the colored blocks.
👆 Click any block in the diagram to see what it does. Notice the arrow from the encoder into the middle of the decoder — that's how the decoder "checks the original meaning."
Press play to translate "The cat sat" → French. The encoder absorbs the meaning; the decoder writes each French word using that meaning.
English → (understand) → French
Encoder understands once; decoder writes word by word.
Input (English)
"The cat sat"
→
Encoder
🧠 understands the meaning
→
Decoder (French)
…
The big reveal
Today they're used separately — GPT vs BERT
Here's the key modern insight: encoders and decoders are now usually used on their own, because each is great at a different job.
Decoder-only → the chatbots
ChatGPT · Claude · Gemini
Job: predict the next word and generate text
Feeds its own output back in (auto-regression) to write long answers
This is what most people interact with — the focus of this course
Encoder-only → the analyzers
Google's BERT
Job: understand a document (not write new text)
Great for sentiment analysis, classification, search
Reads in both directions for deep understanding
The simple rule
Writing text? → you want a decoder (GPT/Claude). Understanding/classifying text? → you want an encoder (BERT). A chatbot writes, so it's a decoder. A spam filter judges, so it's an encoder. From here on, this course focuses on decoder LLMs, since they power the AI you actually talk to.
🎮 Game 2
Encoder or Decoder? Match the task
For each task, pick which type of model fits. Writing = Decoder, Understanding = Encoder.
Pick the right tool for each job
Tap Encoder or Decoder for each.
The whole journey
The Evolution of NLP (a recap of everything)
Straight from your notes — the full arc you've now learned, each model fixing the last one's flaw:
Model
Key feature
Major limitation
Bigram (MLE)
Predicts next word from raw counts of pairs
No context past 1 word; no understanding
word2vec
Words as multi-dimensional vectors (semantics)
Good for analysis, can't predict sequences alone
RNN
Hidden state = memory for previous words
Vanishing gradient: forgets long context
Transformer
Attention: looks at all words at once
Extremely high compute to train
Quick definitions (worth memorizing)
Greedy decoding: always picking the highest-probability word. Softmax: turns raw scores into 0–1 probabilities. Vanishing gradient: when an RNN "forgets" the start of a long sentence. Self-attention: weighing the importance of words regardless of their distance.
🎯 Check your understanding
Quick challenge
1. What is the encoder's job?
2. ChatGPT and Claude are which type?
3. Google's BERT is best for…
Takeaway
What you learned
The original Transformer had two halves: an encoder (the "Understander" — reads all words, outputs a meaning summary) and a decoder (the "Writer" — generates output word by word). Great together for translation. Today they're used separately: decoder-only models (ChatGPT, Claude, Gemini) generate text — the chatbots you use; encoder-only models (BERT) understand/classify text. Simple rule: writing → decoder, understanding → encoder. This course now focuses on decoder LLMs.
Next up → Module 12: Inside the Decoder. We'll follow a single sentence — "The robot learns" — through every stage of the decoder "factory": tokenize → embed → positional encoding → decoder layers → output → auto-regression.