XP: 0
⚙️ Part 2 · Module 9

Embeddings for Search

Half 1 built models. Half 2 builds systems around them. First stop: embeddings as a search tool. You met embeddings as meaning-vectors (fundamentals Module 4) — now we use that to find text by meaning, not keywords. This is the foundation of vector databases and RAG.
Concept → tool

You already know embeddings — now weaponize them

Recall the big idea: an embedding turns text into a vector of numbers, where similar meanings land close together (fundamentals Module 4, "king − man + woman ≈ queen"). Half 1 used embeddings inside a model. Now we use them directly: embed a bunch of documents, embed a query, and find the documents whose vectors are closest to the query's.

The whole idea in one sentence

Closeness in vector space = closeness in meaning. So "find similar text" becomes "find nearby vectors" — a geometry problem a computer can solve instantly, even across millions of documents.

Why it's better

Keyword search vs semantic search

Old search matched exact words. If you searched "car" it missed "automobile." Semantic (embedding) search matches meaning — it knows those are the same idea.

🔤 Keyword search (old)

query: "how to fix my bike"
✕ misses: "repairing a bicycle"
✕ misses: "cycle won't work"
→ only matches literal words

🧠 Semantic search (embeddings)

query: "how to fix my bike"
✓ finds: "repairing a bicycle"
✓ finds: "cycle won't work"
→ matches meaning, any wording
🎮 Game 1

Search the same docs both ways

Type a query. Keyword search only lights up docs sharing your exact words; semantic search finds docs with the same meaning. See the difference.

Keyword 🔤 vs Semantic 🧠
Try: "fix my bike", "cold weather", "feeling sad"
🔤 KEYWORD RESULTS
🧠 SEMANTIC RESULTS
🎮 Game 2

The meaning map (embeddings in 2D)

Real embeddings have hundreds of dimensions, but we can squish them to 2D to see the geography of meaning. Related words cluster. Click any point — its nearest neighbors light up. That "nearest neighbor" lookup is semantic search.

Nearby = similar meaning
Click a word to find its neighbors.
👆 click any word
The math of "close"

Cosine similarity — measuring closeness

How do we measure "how close" two vectors are? The standard tool is cosine similarity: the cosine of the angle between them. Same direction → 1.0 (identical meaning). Perpendicular → 0 (unrelated). Opposite → −1. We care about direction, not length, which is why it's the angle.

python · cosine similarity
import torch, torch.nn.functional as F

a = embed("how to fix my bike")
b = embed("repairing a bicycle")

sim = F.cosine_similarity(a, b, dim=0)
print(sim)   # tensor(0.89) → very similar!
🎮 Game 3

Rotate the vectors, watch the similarity

Drag the angle between two vectors and watch cosine similarity change. When they point the same way, sim = 1. As they spread apart, it drops. This is exactly how "how similar?" is scored.

Angle between two meaning-vectors
angle = 30°
🎮 Game 4

Full semantic search (ranked by score)

The real thing: embed the query, compute cosine similarity to every document, sort by score, return the top matches. Pick a query and watch the ranked results.

Embed query → score all docs → rank
Choose a query.
python · semantic search
from sentence_transformers import SentenceTransformer, util

model = SentenceTransformer("all-MiniLM-L6-v2")   # a small embedding model
doc_embs = model.encode(documents)              # embed all docs once

q_emb = model.encode("how to fix my bike")     # embed the query
scores = util.cos_sim(q_emb, doc_embs)[0]        # similarity to each
top = scores.argsort(descending=True)[:3]        # best 3
🎮 Game 5

Which is more similar?

Build your intuition for semantic closeness. For each query, pick which candidate an embedding model would rank as most similar in meaning.

Pick the closest in meaning
Question 1 of 4
🎮 Game 6

The embedding-search pipeline

Order the steps of a semantic search system, from documents to results. This exact pipeline is what a vector database (next module!) automates for you.

Build the search pipeline in order
Click the steps 1 → 5 in order.
🎯 Check your understanding

Quick challenge

1. What does semantic search match on?
2. In embedding space, similar meanings are…
3. What does cosine similarity measure?
4. What's the first step of semantic search?
5. A cosine similarity of 0.89 between two texts means…
Takeaway

What you learned

Embeddings turn text into vectors where closeness = similar meaning. That powers semantic search: embed your documents once, embed a query, and find the nearest vectors — matching meaning, not keywords. Closeness is scored with cosine similarity (the angle between vectors: 1 = same, 0 = unrelated). The pipeline: embed docs → embed query → score with cosine → rank → return top matches. Libraries like sentence-transformers do the embedding.

Next up → Module 10: Vector Databases (ChromaDB) — computing cosine similarity against millions of docs one-by-one is too slow. A vector database stores embeddings and finds nearest neighbors instantly. You'll use ChromaDB to build a real searchable store.