Chroma Vector Database Tutorial: A Hands-On Guide
Chat2DB TeamChroma is an open-source vector database designed to get out of your way. It runs embedded in your Python process, generates embeddings for you if you have not chosen a model, and stores everything on the local filesystem. For building a semantic search prototype or a small RAG application, it is the shortest path from zero to working retrieval.
This tutorial builds a working document search system, then covers the parts that matter when you move beyond a notebook.
Installation
pip install chromadbThat single package includes the database, a default embedding model, and the client. No server to start, no Docker container, no connection string.
Your first collection
Chroma organises data into collections — roughly analogous to tables. Each entry has an ID, an optional document (the text), an embedding, and optional metadata.
import chromadb
client = chromadb.Client() # in-memory, lost on exit
collection = client.create_collection(name="docs")
collection.add(
documents=[
"PostgreSQL VACUUM reclaims space from dead tuples left by UPDATE and DELETE.",
"PgBouncer pools connections so many clients share few PostgreSQL backends.",
"A B-tree index accelerates equality, range and ORDER BY queries.",
"GIN indexes suit arrays, JSONB containment and full-text search.",
],
metadatas=[
{"topic": "maintenance", "level": "intermediate"},
{"topic": "performance", "level": "intermediate"},
{"topic": "indexing", "level": "beginner"},
{"topic": "indexing", "level": "advanced"},
],
ids=["d1", "d2", "d3", "d4"],
)
results = collection.query(
query_texts=["how do I free up disk space in my database"],
n_results=2,
)
for doc, dist in zip(results["documents"][0], results["distances"][0]):
print(f"{dist:.4f} {doc}")Notice what you did not do: no embedding model was chosen, no vectors were computed by hand, no index was configured. Chroma downloaded a default model (all-MiniLM-L6-v2, a 384-dimension sentence transformer) and embedded everything automatically.
The query returns the VACUUM document first, despite the query sharing no words with it — that is semantic retrieval working.
Persisting data
The in-memory client loses everything when the process exits. For anything real, use PersistentClient:
import chromadb
client = chromadb.PersistentClient(path="./chroma_data")
collection = client.get_or_create_collection(name="docs")get_or_create_collection is the idempotent form — it will not raise if the collection already exists, which makes scripts re-runnable.
Data now lives in ./chroma_data as a SQLite database plus index files. Back that directory up and you have backed up your vector store.
Choosing an embedding model
The default model is small and fast but not especially strong. For production, pick a model deliberately.
OpenAI embeddings
from chromadb.utils import embedding_functions
openai_ef = embedding_functions.OpenAIEmbeddingFunction(
api_key="sk-...",
model_name="text-embedding-3-small",
)
collection = client.get_or_create_collection(
name="docs_openai",
embedding_function=openai_ef,
metadata={"hnsw:space": "cosine"},
)A local sentence-transformer
st_ef = embedding_functions.SentenceTransformerEmbeddingFunction(
model_name="BAAI/bge-base-en-v1.5"
)Bringing your own vectors
If you compute embeddings elsewhere, pass them directly:
collection.add(
ids=["d5"],
embeddings=[[0.021, -0.114, 0.335]], # your vector
documents=["Logical replication decodes WAL into row-level changes."],
metadatas=[{"topic": "replication"}],
)One rule matters above all: the embedding function must be identical for indexing and querying. Mixing models produces vectors in incompatible spaces and results that look random. Because Chroma stores the embedding function with the collection, always retrieve a collection with the same embedding_function you created it with:
collection = client.get_collection(name="docs_openai", embedding_function=openai_ef)Changing embedding models means re-embedding the entire collection. There is no migration path — the old vectors are simply meaningless in the new space.
Metadata filtering
Filtering is what turns a vector store into an application backend. Chroma applies filters with a MongoDB-like syntax.
# Simple equality
collection.query(
query_texts=["index performance"],
n_results=3,
where={"topic": "indexing"},
)
# Operators
collection.query(
query_texts=["database tuning"],
n_results=5,
where={"level": {"$in": ["beginner", "intermediate"]}},
)
# Combining conditions
collection.query(
query_texts=["database tuning"],
n_results=5,
where={
"$and": [
{"topic": {"$eq": "indexing"}},
{"level": {"$ne": "advanced"}},
]
},
)Supported operators include $eq, $ne, $gt, $gte, $lt, $lte, $in, $nin, plus $and and $or for composition.
You can also filter on document content directly:
collection.query(
query_texts=["connection management"],
n_results=3,
where_document={"$contains": "PgBouncer"},
)A caveat on filtered search
Filters constrain the candidate set, which means a restrictive filter combined with a small n_results can return fewer results than you expect, or force a much wider scan. If you are filtering to a small subset of a large collection, consider a separate collection per tenant or category instead — collections are cheap.
Updating and deleting
# Update the text and metadata for an existing ID (re-embeds automatically)
collection.update(
ids=["d1"],
documents=["VACUUM FULL rewrites a table and returns space to the OS."],
metadatas=[{"topic": "maintenance", "level": "advanced"}],
)
# Insert or update in one call
collection.upsert(
ids=["d6"],
documents=["CREATE INDEX CONCURRENTLY avoids blocking writes."],
metadatas=[{"topic": "indexing", "level": "advanced"}],
)
# Delete by ID
collection.delete(ids=["d3"])
# Delete by filter
collection.delete(where={"level": "beginner"})Inspecting a collection
print(collection.count())
# Peek at the first few entries
print(collection.peek(limit=3))
# Fetch specific entries with their embeddings
print(collection.get(ids=["d1"], include=["documents", "metadatas", "embeddings"]))
# List all collections
print(client.list_collections())By default get and query omit embeddings from the response because they are large; request them explicitly with include.
Client/server mode
Embedded mode means one process owns the data. To share a store across services, run Chroma as a server:
chroma run --path ./chroma_data --host 0.0.0.0 --port 8000Or with Docker:
docker run -p 8000:8000 -v $(pwd)/chroma_data:/chroma/chroma chromadb/chromaThen connect over HTTP:
client = chromadb.HttpClient(host="localhost", port=8000)
collection = client.get_or_create_collection(name="docs")The API is identical, so code written against the embedded client works unchanged against the server.
A complete example
Putting it together into a small document search tool:
import chromadb
from chromadb.utils import embedding_functions
ef = embedding_functions.SentenceTransformerEmbeddingFunction(
model_name="BAAI/bge-base-en-v1.5"
)
client = chromadb.PersistentClient(path="./kb_store")
collection = client.get_or_create_collection(
name="knowledge_base",
embedding_function=ef,
metadata={"hnsw:space": "cosine"},
)
def chunk(text: str, size: int = 300, overlap: int = 40) -> list[str]:
words = text.split()
step = size - overlap
return [" ".join(words[i:i + size]) for i in range(0, len(words), step)]
def index_document(doc_id: str, text: str, source: str) -> None:
chunks = chunk(text)
collection.upsert(
ids=[f"{doc_id}::{i}" for i in range(len(chunks))],
documents=chunks,
metadatas=[{"source": source, "doc_id": doc_id, "chunk": i}
for i in range(len(chunks))],
)
def search(query: str, k: int = 5, source: str | None = None):
where = {"source": source} if source else None
res = collection.query(query_texts=[query], n_results=k, where=where)
return [
{"text": d, "score": 1 - dist, "source": m["source"]}
for d, dist, m in zip(res["documents"][0],
res["distances"][0],
res["metadatas"][0])
]
index_document("pg-vacuum", open("vacuum.md").read(), "postgres-docs")
for hit in search("how do I stop tables from bloating", k=3):
print(f"{hit['score']:.3f} [{hit['source']}] {hit['text'][:100]}...")Note the ID scheme: {doc_id}::{chunk_index}. Because upsert is idempotent on ID, re-indexing an edited document overwrites its chunks cleanly rather than accumulating duplicates. If the edited version has fewer chunks than before, delete the stale tail first:
collection.delete(where={"doc_id": "pg-vacuum"})
index_document("pg-vacuum", new_text, "postgres-docs")When to move on from Chroma
Chroma is excellent for prototypes, internal tools and collections up to a few million vectors. Consider moving when you hit one of these:
- Concurrency — heavy simultaneous writes from many services.
- Scale — tens of millions of vectors, where memory-efficient quantisation matters.
- Transactional consistency — your vectors must stay in sync with relational data. Here
pgvectorinside PostgreSQL is a natural fit, since embeddings and business data live in the same transaction. - Operational maturity — you need replication, point-in-time recovery and established backup tooling.
Qdrant and Milvus are the usual next steps for self-hosted scale; pgvector is the answer when consistency with existing relational data matters more than raw vector throughput.
Wrapping up
Chroma earns its popularity by removing decisions: no server, no index tuning, no embedding pipeline required to see your first result. Use that to validate whether semantic search solves your problem at all, then make the harder infrastructure choices with real data in hand.
If your next step is moving embeddings into PostgreSQL with pgvector, Chat2DB (opens in a new tab) connects to PostgreSQL and lets you inspect vector columns, check index definitions and write similarity queries — including generating the SQL from a plain-English description.