Part D · Module 12

Inside the Decoder

Time to open the "factory." We'll follow one sentence — "The robot learns" — through every stage of a decoder LLM: from raw text to a predicted next word, then looping to write more. Six stages, each with a memory trick.

The assembly line

The 6 stages of the decoder "factory"

A decoder LLM turns text into a prediction through a fixed pipeline. Here's the whole thing at a glance — each stage with the memory trick from your notes:

🪓

1. Tokenization — the Slicer

Break text into tokens — chunks of words, numbers, punctuation. "learns" → ["learn", "s"]. Sub-word splitting helps meaning (think how "un" flips a word).

🧠 Lego bricks: smash the sentence into its smallest reusable building blocks.

📖

2. Embedding — the Dictionary

Each token becomes a vector (hundreds/thousands of numbers) capturing its meaning — exactly like word2vec (Module 5). "robot" → "mechanical, object, tech".

🧠 GPS coordinate for meaning: "robot" and "machine" land near each other on the meaning-map.

🔢

3. Positional Encoding — the Timestamp

Transformers see all words at once, so they have no idea of order. We add a position signal (1, 2, 3…) so the model knows the sequence.

🧠 Seat numbers at a theater: without them, words are a crowd; with them, a structured row.

🧠

4. Decoder Layers — the Brain

A stack of many identical decoder blocks (same design, different weights). Each does: self-attention (tokens talk), feed-forward (a small NN), and normalization (keep values balanced).

🧠 Spotlight: "learns" shines a light on "robot" → "I'm the action the robot does!"

🏇

5. Output Layer — the Probability

The final layer's last token is mapped to the whole vocabulary, producing a probability for every possible next word. "fast": 80%, "slowly": 15%, "banana": 0.001%.

🧠 Horse race: every word in the dictionary is running; "fast" wins.

🧶

6. Auto-Regression — the Loop

The winning word is added back to the input, and the whole process runs again — word by word — until an <EOS> (end-of-sequence) token stops it.

🧠 Knitting: each new stitch depends on the last; the scarf grows until you tie it off.

🎮 Game 1 — the main event

Run "The robot learns" through the factory

Step through each stage and watch the sentence transform — sliced into tokens, turned into vectors, timestamped, understood, then raced to a prediction.

The Transformer factory, stage by stage

Press "Next stage" to begin.

Stage 0 — raw input

"The robot learns"

🎮 Zoom in

Inside ONE decoder block (click each part)

Stage 4 said "a stack of decoder blocks." But what's inside a single block? Three sub-layers, each wrapped in an "Add & Norm." Data flows bottom → top. Click each part.

The anatomy of one decoder block

This exact block is stacked dozens of times.

👆 Click a sub-layer to see its job. The dashed "Add & Norm" wrappers appear after every sub-layer — they keep the numbers stable.

🎮 The stack

Why stack many blocks? Each layer learns different things

From your notes: earlier layers focus on short, local connections; later layers pick up global, abstract, long-range meaning. Click each layer (bottom = first) to see what it specializes in.

Climbing the decoder stack (bottom → top)

Same architecture, different weights, different focus.

👆 Click any layer to see what it learns.

🎮 Game 2

The auto-regression loop — writing a full sentence

One pass predicts one word. To write a whole sentence, the model loops: predict → append → predict again. Watch it build "The robot learns fast." one word at a time, then stop at <EOS>.

Predict → append → repeat → <EOS>

Each cycle feeds its own output back as input (the "knitting").

The big comparison

RNN vs. Transformer — side by side

Now you can see the full contrast (straight from your notes). This table sums up why the Transformer won:

Feature	RNN	Transformer (decoder LLM)
Speed	Sequential — must finish "The" before "robot"	Parallel — all words enter at once
Memory	Hidden state (a blurry notepad, fades)	Self-attention (a spotlight, no distance limit)
Order	Natural — receives words in order	Artificial — needs positional encoding added
Next word	From the final blurry note	From the entire context at once

The two experiences, in a nutshell

RNN: reads "The" → "robot" (tries to remember "The") → "learns" (starts forgetting "The") → predicts from a messy blend. Transformer: takes all three at once → adds positions → "learns" spotlights "robot" → instantly sees the full picture → predicts "fast" confidently.

🎯 Check your understanding

Quick challenge

1. What is tokenization?

2. Why is positional encoding needed?

3. What happens inside each decoder layer?

4. What is auto-regression?

Takeaway

What you learned

A decoder LLM is a 6-stage factory: Tokenize (Lego bricks) → Embed (GPS for meaning) → Positional encoding (seat numbers) → Decoder layers (the brain: self-attention + feed-forward + normalization, stacked many times) → Output layer (a horse race of probabilities) → Auto-regression (knitting: append the winner, loop until <EOS>). Unlike an RNN, it processes all words in parallel, uses attention instead of a fading memory, and needs positional encoding because it has no natural sense of order.

Next up → Module 13: Running LLMs Yourself (the finale!). Now that you understand what's inside, let's use one: self-hosting with Ollama, the all-important system prompt, and the two creativity dials — temperature and top-p. Plus a wrap-up of activation functions.

🧭 Course Map ← Module 11 Continue to Module 13: Running LLMs →