Module 8 left us with a broken RNN: its memory fades over distance. Attention's insight is radical — don't carry a summary forward at all. Instead, when processing any word, let it reach back and look directly at every earlier word, and decide which ones matter right now.
Picture a stage with 100 actors (the words). An RNN only has a scribbled summary of the play so far — and it's forgotten the early scenes. But an LLM with attention can shine a spotlight directly on any actor, no matter how far back they are. Processing word #101? Shine the light straight on word #3 if that's what matters. No fading, no burial — a direct connection to any word.
An RNN is like listening to a story through a chain of whisperers (telephone game — it degrades). Attention is like being in the room with everyone: when someone says "it," you instantly glance at whoever they're referring to, however long ago they spoke. You attend to the relevant person directly.
Click any word to make it the query (the word doing the looking). Watch it "attend" to the other words — the ones it cares about light up brightest, with an attention score. This is the entire mechanism: every word decides how much to focus on the words it's allowed to see.
Attention runs in one of two modes depending on the job:
✍️ Writing mode — the model is generating text one word at a time (like a chatbot typing a reply). At each word, the words after it don't exist yet, so a word can only look backward. This "no peeking at the future" rule is called causal masking.
📖 Reading mode — the model is handed a complete, existing sentence to understand (like analysing a review). Since the whole sentence is already there, every word can look both ways.
Toggle between them below and watch which words a query is allowed to attend to. (Chatbots use writing mode — we'll name these model types properly in Module 11.)
This is called self-attention: the sentence attending to itself, every word weighing every other word it's allowed to see. In writing mode, try clicking "because" — the words "was" and "hungry" are greyed out 🔒 because they haven't been written yet. Now switch to reading mode and click it again: now it can see them, because the whole sentence already exists.
Great question to ask. Back in Module 6, bigrams had probabilities like P(sat|cat)=0.40 — those are weights. Attention also has weights. So why did attention change everything? Because the two kinds of "weight" measure completely different things.
Remember river bank vs. money bank? N-gram: P(?|bank) is the identical frozen number both times — it literally cannot tell them apart. Attention: "bank" attends to "river" in one sentence and "money" in the other, so it gets a different meaning each time. That's the leap: from a frozen lookup table to live, context-aware understanding.
An n-gram is a phone book: fixed entries, looked up the same way every time ("after 'cat' → usually 'sat'"). Attention is a smart assistant in the room: when you say a word, it instantly glances at whatever else you've said that's relevant — and its focus shifts depending on the whole conversation. One recites frozen facts; the other actually follows the meaning.
This landmark paper made a bold claim: you don't need recurrence (RNNs) at all — attention alone is enough to model language. It introduced the Transformer architecture, which is now at the core of almost every modern AI system you've used — ChatGPT, Claude, Gemini are all built on Transformers. (An LLM, "Large Language Model," is just a very big Transformer trained on huge amounts of text.)
For years, everyone assumed you needed recurrence to handle sequences. The paper's title is a direct challenge: attention is ALL you need — drop the RNN entirely. It was right, and it triggered the entire modern AI boom. Every chatbot you've used descends from this 2017 paper.
Removing recurrence didn't just fix memory — it unlocked two massive advantages that made today's giant models possible:
RNN: can't process word #10 until #9, #8, #7 are done — strictly one at a time, painfully slow. Transformer: looks at all words simultaneously. This is what lets us train on the entire internet in reasonable time.
RNN: the canvas covers up old words. Transformer: no canvas — it attends to all words directly, so the first word of a book is just as accessible as the last. Distance stops mattering.
Press Race and watch the difference in speed. The RNN must process words one-by-one (each waits for the last). The Transformer processes them all at once. This parallelism is why modern models can be trained at all.
Your notes' killer example: "The dog, which had been running through the park all morning chasing squirrels and playing with children, finally found its bone." An RNN buries "dog" under all that middle noise. Attention leaps straight over it. Watch both.
Next up → Module 10: Query, Key, Value ★. We've seen what attention does (the spotlight). Now we open the hood: how does a word decide what to attend to? The answer is a beautifully simple search mechanism — Query, Key, and Value.