|$ curl https://forge-ai.dev/api/markdown?path=docs/ai/vector-dbs

$cat docs/vector-databases.md

updated Recently·35 min read·published

Vector Databases

◆Vector DB◆AI◆Infrastructure◆Intermediate

What is a Vector Database?

A vector database is a specialized database designed to store, index, and query vector embeddings. Unlike traditional databases that search by exact or structured queries, vector databases retrieve data by semantic similarity — finding vectors that are nearest to a query vector in high-dimensional space.

Vector databases are the backbone of modern RAG systems. They enable fast approximate nearest neighbor (ANN) search over millions or billions of vectors, making semantic search practical at scale. Each vector in the database corresponds to an embedded document chunk, along with its original text and metadata.

ℹ

info

The key difference between a vector database and a vector index library (like FAISS) is database features: CRUD operations, filtering, hybrid search, replication, and persistence. Use a vector database for production; use FAISS for local prototyping and batch processing.

Indexing Methods

The indexing algorithm determines how vectors are organized for fast search. The choice of index directly impacts search speed, memory usage, and recall. Most vector databases offer multiple index types.

HNSW (Hierarchical Navigable Small World)

HNSW is the most popular graph-based index. It builds a multi-layer graph where higher layers have fewer nodes (long-range connections) and lower layers have more nodes (short-range connections). Search starts at the top layer and descends toward the bottom, navigating to closer neighbors at each level.

Parameter	Effect	Tradeoff
M	Number of bi-directional connections per node	Higher M = better recall, more memory
efConstruction	Dynamic list size during index building	Higher = better index quality, slower build
efSearch	Dynamic list size during search	Higher = better recall, slower search

hnsw_config.py

Python

1	# Configuring HNSW in Chroma
2	collection = client.create_collection(
3	name="hnsw_demo",
4	metadata={
5	"hnsw:space": "cosine",
6	"hnsw:construction_ef": 200,
7	"hnsw:M": 32,
8	"hnsw:search_ef": 100,
9	}
10	)

IVF (Inverted File Index)

IVF partitions the vector space into regions using k-means clustering. At search time, only the clusters closest to the query are searched. IVF is more memory-efficient than HNSW and can be combined with other index types (IVF+Flat, IVF+PQ).

ivf_config.py

Python

1	# IVF configuration in Milvus
2	from pymilvus import CollectionSchema, FieldSchema, DataType, IndexType
3
4	index_params = {
5	"index_type": IndexType.IVF_FLAT,
6	"metric_type": "IP", # Inner product
7	"params": {"nlist": 1024}
8	}
9
10	# nlist controls the number of clusters
11	# Search parameter nprobe controls clusters to visit:
12	search_params = {
13	"metric_type": "IP",
14	"params": {"nprobe": 16}
15	}

Flat (Brute Force)

The Flat index performs an exact, exhaustive search — comparing the query vector against every stored vector. It provides perfect recall but does not scale to large datasets. Use Flat for small datasets or as a baseline for evaluating approximate methods.

flat_index.py

Python

1	# FAISS Flat index (exact search)
2	import faiss
3
4	dimension = 768 # e.g., text-embedding-3-small
5	index = faiss.IndexFlatIP(dimension)
6
7	# Add vectors
8	index.add(embeddings)
9
10	# Search (k = number of results)
11	distances, indices = index.search(query_embedding, k=5)
12	# Exact results — no approximation error

📝

note

Flat search is O(n) in both time and memory. For datasets over 100k vectors, use an approximate index. Use Flat to validate your accuracy requirements, then tune approximate indices to match that baseline.

Similarity Metrics

The similarity metric defines how distance between vectors is measured. The choice of metric should match your embedding model's training objective.

Metric	Formula	Range	Use Case
Cosine Similarity	cos(θ) = A·B / \|A\|\|B\|	[-1, 1]	Most embedding models (OpenAI, Cohere)
Dot Product	A·B = Σ A_iB_i	(-∞, ∞)	Normalized embeddings, inner product models
Euclidean (L2)	d(A,B) = √ Σ(A_i-B_i)²	[0, ∞)	Spatial clustering, anomaly detection
Manhattan (L1)	d(A,B) = Σ\|A_i-B_i\|	[0, ∞)	High-dimensional sparse vectors

✓

best practice

Cosine similarity and dot product are equivalent for normalized vectors (|A| = |B| = 1). Most modern embedding models produce normalized vectors, so using dot product (IP) with normalized embeddings is computationally identical to cosine similarity but faster to compute.

Filtering and Hybrid Search

Real-world applications need more than pure vector search. Metadata filtering restricts search to a subset of documents, while hybrid search combines vector similarity with keyword matching for better results.

Pre-Filtering

Filter by metadata before performing vector search. This narrows the search space and improves both speed and relevance.

pre_filtering.py

Python

1	# Pre-filtering with metadata in Pinecone
2	index.query(
3	vector=query_embedding,
4	filter={"category": {"$eq": "documentation"},
5	"date": {"$gte": "2025-01-01"}},
6	top_k=10,
7	include_metadata=True
8	)
9
10	# In Weaviate with GraphQL
11	{
12	Get {
13	Document(
14	nearVector: { vector: query_embedding }
15	where: {
16	operator: And
17	operands: [
18	{ path: ["category"], operator: Equal, valueString: "docs" }
19	{ path: ["date"], operator: GreaterThan, valueDate: "2025-01-01" }
20	]
21	}
22	limit: 10
23	) {
24	title
25	content
26	}
27	}
28	}

Post-Filtering

Retrieve a larger pool of candidates via vector search, then apply metadata filters to the results. This is simpler but less efficient than pre-filtering.

post_filtering.py

Python

1	# Post-filtering: retrieve many, then filter
2	results = collection.query(
3	query_embeddings=[query_embedding],
4	n_results=100, # Larger pool
5	include=["documents", "metadatas"]
6	)
7
8	# Filter after retrieval
9	filtered = [
10	(doc, meta) for doc, meta
11	in zip(results["documents"][0], results["metadatas"][0])
12	if meta["category"] == "documentation"
13	][:10]

ℹ

info

Pre-filtering is more performant because the index prunes before search. However, aggressive pre-filtering can exclude relevant results if the metadata schema is incomplete. A common pattern is pre-filtering with broad criteria followed by post-filtering for finer-grained constraints.

Vector Database Comparison

Choosing the right vector database depends on your scale, latency requirements, and feature needs. Here is a comparison of the most popular options.

Database	Open Source	Cloud	Index Types	Filtering	Hybrid
Pinecone	No	Yes	HNSW (managed)	Pre/post	Sparse-dense
Weaviate	Yes (BSD-3)	Yes	HNSW	Pre-filter	BM25 + vector
Qdrant	Yes (Apache 2)	Yes	HNSW, custom	Pre/post	BM25 + vector
Chroma	Yes (Apache 2)	No	HNSW	Pre-filter	Limited
Milvus	Yes (Apache 2)	Yes (Zilliz)	IVF, HNSW, Flat, PQ	Pre/post	BM25 + vector
Pgvector	Yes (PostgreSQL)	Via pgvector	IVF, HNSW	SQL filters	TSVector + vector

✓

best practice

Start with Chroma for prototyping (local, zero-config), then migrate to Pinecone or Weaviate for production. If you already use PostgreSQL, pgvector eliminates the operational cost of managing a separate database. Milvus is the best choice for large-scale (100M+ vectors) deployments.

Key Tradeoffs

✗ Tradeoff

Recall vs. Latency

Approximate indices trade perfect recall for speed. Tune efSearch (HNSW) or nprobe (IVF) to find your acceptable recall-latency balance.

✗ Tradeoff

Memory vs. Accuracy

Flat index: high memory, exact recall. Product Quantization (PQ): 4-16x compression, slight accuracy loss.

✗ Tradeoff

Build Time vs. Search Speed

HNSW builds slowly but searches fast. IVF builds faster but may be slower at search depending on nprobe.

$Blueprint — Engineering Documentation·Section ID: AI-VECTOR-DBS·Revision: 1.0

X GitHub YouTube Instagram Facebook Buy me a coffee Sponsor