Vector Databases (ChromaDB)

Comparing to every doc doesn't scale

Module 9's approach was a brute-force scan: to find the closest vector, compare the query to all of them. With 8 docs that's instant. With 8 million docs — and thousands of queries a second — it collapses. A vector database solves this: it indexes your vectors so it can find the nearest ones without checking them all.

Brute-force vs vector DB — the speed race

Slide the number of documents and race the two approaches. Brute force scales with the collection size; the vector DB stays nearly flat thanks to its index.

What's inside: vectors + IDs + metadata + documents

A vector DB stores four things per item: the embedding vector (for similarity), an ID, the original document text, and optional metadata (tags like author, date, category — for filtering). You add items once; then you query by meaning.

import chromadb
client = chromadb.Client()
collection = client.create_collection("my_docs")

collection.add(
    documents=["Repairing a bike tire", "Best winter coats"],
    metadatas=[{"topic":"bikes"}, {"topic":"clothing"}],
    ids=["doc1", "doc2"],
)   # Chroma embeds the text for you automatically!

Add documents to a collection

Click "Add" to insert docs one by one. Watch each get an auto-generated embedding vector, an ID, and its metadata — exactly what collection.add() does.

Query it — nearest neighbors, instantly

Now the payoff. Type a query; the DB returns the top-k most similar documents by meaning — in one call. This is what you built manually in Module 9, but instant and scalable.

results = collection.query(
    query_texts=["fix a bicycle"],
    n_results=3,               # top 3 nearest
)
print(results["documents"])   # the 3 most similar docs

Filter by metadata (search + constraints)

A key power: combine semantic search with exact filters. "Find docs similar to X, but only where topic = bikes and year ≥ 2023." Toggle filters and watch the results narrow.

results = collection.query(
    query_texts=["how do I fix this"],
    n_results=3,
    where={"topic": "bikes"},   # metadata filter!
)

How is it so fast? Approximate Nearest Neighbors

The trick behind vector DBs: instead of comparing the query with every stored vector, an index identifies a promising region or route and scores only a candidate set. It usually finds a very-nearly nearest result while doing far less work. This is ANN — Approximate Nearest Neighbor search.

ANN: hop through a graph, don't scan everything

Watch the difference. Exact search checks every point (red, slow). ANN hops through a small graph toward the query (teal, fast) — visiting a handful of points instead of all of them.

Is ChromaDB only in memory? No — these are deployment modes

Chroma is one database with several ways to run or connect to it. In-memory is convenient for a disposable notebook, while persistent, server, and cloud modes keep data beyond one Python process.

import chromadb

memory = chromadb.Client()
local = chromadb.PersistentClient(path="./chroma_data")
server = chromadb.HttpClient(host="localhost", port=8000)
cloud = chromadb.CloudClient(
    tenant="tenant-id", database="database-name", api_key="..."
)

Six common choices — and what actually differs

A product choice is not merely “which one is fastest?” Consider where it runs, who operates it, existing infrastructure, filtering and hybrid-search needs, index choices, update rate, multi-tenancy, scale, and cost.

A vector database is more than “store and search”

Which vector DB, and when?

Chroma is great for learning & small apps, but there's a whole ecosystem. Match each situation to a sensible choice.

Quick challenge

What you learned

Next up → Module 11: RAG (Retrieval-Augmented Generation) — the big payoff of Half 2. Combine your vector DB with an LLM: retrieve relevant docs, stuff them into the prompt, and let the model answer using your data. This is how you give an LLM knowledge it was never trained on.