Interactive simulator to build a text-generating AI model stage by stage: dataset, tokenization, embeddings, architecture, training, and generation.
1 Dataset
2 Tokenize
3 Embed
4 Architecture
5 Train
6 Generate
⭐ 0 XP
🏗️ Capstone · Module 14 · The Build Simulator
Build an AI — from raw data to a working model
You've learned every piece. Now assemble them. This is a free sandbox: work through the six stages in any order, watch each one feed the next, and end with a tiny text model that actually generates. The more stages you complete, the smarter your model gets.
💡 Everything here is a simulation to make the concepts visible — the training is animated, not real ML in your browser. But every step mirrors exactly what a real pipeline does.
📚
1. Get a Dataset
not started
Every model learns from data. Pick what your model will learn to write like. This is the "experience" it studies.
💬 Why it matters: a model can only learn patterns that exist in its data. Feed it recipes, it writes recipes. Feed it poems, it writes poems. Garbage in → garbage out.
🔎 What am I looking at?
Three example datasets — collections of text your model will learn from. Click one, then "Load" to see a sample (each sentence ends with an <EOS> = end-of-sentence marker). Whatever you pick is the only thing your model will know how to write.
🪓
2. Tokenize
not started
A model can't read letters — it reads tokens. Slice the dataset into a vocabulary of chunks and add special tokens.
💬 Why it matters: tokens are the atoms the model works with. <EOS> teaches it where sentences end so it knows when to stop writing. (Module 12's "Lego bricks".)
🔎 What am I looking at?
After you build it, you'll see the vocabulary — every unique chunk ("token") the model is allowed to use, shown as chips. Purple chips are special tokens like <EOS>. The model can only ever output words from this list.
← Load a dataset first, then build the vocabulary.
📍
3. Create Embeddings
not started
Turn each token into a vector — a list of numbers capturing its meaning, so related words sit close together. (Module 4's word2vec idea.)
💬 Why it matters: embeddings let the model do math with meaning — this is why "king − man + woman ≈ queen" works. Numbers, not words, are what the network actually processes.
🔎 What am I looking at?
Each word becomes a row of colored bars — that bar IS the word turned into 8 numbers (its "embedding"). The height/shade of each bar = one number.
Words that mean similar things get similar bars (grouped & labeled for you).
The two groups look clearly different — that difference is the model understanding meaning.
Why numbers? The network can't do math on letters — only on numbers.
← Build a vocabulary first.
🏛️
4. Choose the Architecture
not started
Pick the "brain design." Your course taught why one of these wins — choose and set its size.
💬 Why it matters: the architecture decides how the model handles context. Attention (Transformer) reads all words at once with no memory loss — the reason modern AI works (Modules 9–13).
🔎 What am I looking at?
You're designing the model's brain. First pick the type (the two cards), then set its size with the sliders:
Decoder layers = how many "thinking blocks" are stacked. More = deeper understanding (Module 12's stack).
Attention heads = how many parallel "specialists" read the sentence at once (Module 10's jigsaw friends).
Parameters = the total "knobs" the model can tune. Bigger = more capacity to learn.
In the diagram below: each orange box = one decoder block · purple dots = its attention heads · "FF" = its feed-forward network. Data flows bottom → top.
🏋️
5. Train the Model
not started
Now the model studies your data over and over, adjusting its weights to lower the loss (error). Watch it learn.
💬 Why it matters: this is backpropagation from Module 3 — "turning the knobs." Loss dropping = the model getting better at predicting the next token. Accuracy climbing = it's working.
🔎 What am I looking at?
Press start and watch two lines race: the red loss line should fall (fewer mistakes) and the teal accuracy line should rise (more correct guesses). Each pass over the data is an epoch. This is the model "turning its knobs" to get better — backpropagation from Module 3.
epoch0
loss—
accuracy—
✨
6. Use Your Model
not started
The payoff. Give it a starting word and watch auto-regression write, one token at a time, until <EOS>.
💬 Why it matters: this is inference — the model you built, running. Quality depends on everything above: a well-trained model on good data with attention writes coherently. Skip steps and it babbles.
🔎 What am I looking at?
Pick a starting word and press Generate. The model writes one word at a time, each new word feeding back in to choose the next (auto-regression), until it decides to stop with <EOS>. If you built well, it reads sensibly; if you cut corners, it wobbles.
start with:
Your model's output will appear here…
🎓 You just built an AI, end to end.
Dataset → Tokens → Embeddings → Architecture → Training → Generation. That's the entire pipeline behind every LLM — the same stages, just billions of times bigger. You now understand how AI is built, not just how it's used.