Vector Databases in Practice: Embeddings & Similarity Search

Overview

Vector databases store embeddings (the dense numerical vectors that ML models produce) and answer “which stored vectors are closest to this one” fast enough for production. That single capability powers semantic search, recommendations, image retrieval, and the retrieval step in every RAG pipeline.

This guide is the practical path: how vector search actually works, how to pick between Pinecone, Weaviate, Qdrant, pgvector, Chroma, and Milvus, and working code for your first pgvector table, production operations on the dedicated engines, and a complete RAG pipeline.

When to Use

• You need semantic search — “find things with similar meaning” rather than exact keyword match
• You’re building RAG and need to retrieve the chunks that ground an LLM’s answers
• Recommendations or “more like this” features on user or item embeddings
• Image, audio, or video retrieval by content similarity
• You already have embedding models and need storage that indexes and scales

When not to reach for one: exact-match lookups, joins, and aggregations are relational work. Vector databases complement your primary database; they don’t replace it.

How Vector Search Works

1. Embedding: a model (OpenAI, BERT, CLIP) converts text or images into a dense vector, typically 768–1536 dimensions
2. Indexing: vectors are organized into an ANN index (HNSW, IVF, PQ) so retrieval doesn’t scan every row
3. Query: the query gets embedded with the same model and the index returns its K nearest neighbors
4. Metadata filtering: vector similarity combines with ordinary filters (date, category, tenant ID) to keep results relevant

Comparison

Database	Deployment	Index	Best For
Pinecone	Managed cloud	HNSW, metadata filters	Production RAG, no ops overhead
Weaviate	Self-hosted / cloud	HNSW, BM25 hybrid	Multi-modal, GraphQL interface
pgvector	PostgreSQL extension	ivfflat, hnsw	Teams already on Postgres
Chroma	Embedded / local	HNSW	Prototyping, small-scale local RAG
Milvus/Zilliz	Self-hosted / cloud	IVF, HNSW, GPU	Large scale, high throughput

The sections below cover working code for each of these engines. For the decision itself, jump to Choosing a Vector Database.

pgvector Example

The fastest way to try vector search is inside Postgres you already run. This example uses vector(3) so the literals stay readable; real embeddings are 768–1536 dimensions:

-- Enable pgvector extension
CREATE EXTENSION IF NOT EXISTS vector;

-- Create table with vector column
CREATE TABLE documents (
    id SERIAL PRIMARY KEY,
    title TEXT NOT NULL,
    content TEXT,
    embedding vector(3)
);

-- Create HNSW index for fast ANN search
CREATE INDEX ON documents USING hnsw (embedding vector_cosine_ops);

-- Insert a document (in production this vector comes from your embedding model)
INSERT INTO documents (title, content, embedding)
VALUES ('Vector DB Guide', 'A guide to vector databases', '[0.12,-0.03,0.44]');

-- Semantic search: find the 5 most similar documents
SELECT id, title, content,
    1 - (embedding <=> '[0.11,-0.02,0.43]') AS similarity
FROM documents
ORDER BY embedding <=> '[0.11,-0.02,0.43]'
LIMIT 5;

Hybrid Search (Vector + Keyword)

Pure vector search misses exact terms; pure keyword search misses synonyms. Hybrid search combines both: Weaviate exposes it through a single alpha parameter:

import weaviate

client = weaviate.connect_to_local()

# alpha=0.5 weighs BM25 and vector similarity equally
results = client.collections.get("Article").query.hybrid(
    query="vector database architecture",
    vector=embed_query("vector database architecture"),  # your embedding function
    alpha=0.5,  # 0 = pure BM25, 1 = pure vector
    limit=10
)

Dedicated Vector Databases

pgvector and Chroma cover a lot of ground, but at some point you want an engine built for the job: managed scaling, billions of vectors, or filtering semantics Postgres can’t express. Here are the production snippets for the four dedicated options you’ll meet most often.

Pinecone

Pinecone is the fully managed option: no servers, no index tuning beyond the initial metric, and pay-per-use billing. Two patterns matter in production — batching upserts and namespaces for multi-tenancy:

from pinecone import Pinecone, ServerlessSpec

pc = Pinecone(api_key="your-api-key")

pc.create_index(
    name="products",
    dimension=1536,
    metric="cosine",
    spec=ServerlessSpec(cloud="aws", region="us-east-1"),
)
index = pc.Index("products")

# Upsert in batches: 100 keeps payloads small and retries cheap
def batch_upsert(index, vectors, batch_size=100):
    for i in range(0, len(vectors), batch_size):
        index.upsert(vectors=vectors[i:i + batch_size])

# Multi-tenancy: one index, one namespace per tenant
index.upsert(vectors=tenant_vectors, namespace="tenant_a")

results = index.query(
    vector=query_embedding,
    top_k=10,
    namespace="tenant_a",
    include_metadata=True,
    filter={"category": {"$eq": "electronics"}, "price": {"$lte": 50.0}},
)

Namespaces keep tenant data isolated inside a single index — cheaper than one index per tenant and a hard boundary you can’t get from metadata filters alone.

Qdrant

Qdrant is open source (written in Rust), strong on payload filtering, and available self-hosted or as Qdrant Cloud. Note that client.search() was removed in qdrant-client 1.16 — use query_points instead:

from qdrant_client import QdrantClient
from qdrant_client.models import (
    Distance, VectorParams, PointStruct, Filter, FieldCondition, MatchValue,
)

client = QdrantClient(host="localhost", port=6333)

client.create_collection(
    collection_name="documents",
    vectors_config=VectorParams(size=1536, distance=Distance.COSINE),
)

client.upsert(
    collection_name="documents",
    points=[
        PointStruct(id=i, vector=vec,
                    payload={"text": f"document {i}", "category": "tech"})
        for i, vec in enumerate(embeddings)
    ],
)

results = client.query_points(
    collection_name="documents",
    query=query_embedding,
    query_filter=Filter(
        must=[FieldCondition(key="category", match=MatchValue(value="tech"))]
    ),
    limit=10,
).points

for result in results:
    print(result.score, result.payload)

Milvus

Milvus is built for scale — billions of vectors, GPU indexing, distributed deployments — with Zilliz as its managed cloud. The quick-setup create_collection only takes dimension and metric_type; for an explicit index, build IndexParams first:

from pymilvus import MilvusClient

client = MilvusClient(uri="http://localhost:19530")

# Explicit HNSW index via prepare_index_params
index_params = client.prepare_index_params()
index_params.add_index(
    field_name="vector",
    index_type="HNSW",
    metric_type="COSINE",
    params={"M": 16, "efConstruction": 64},
)
client.create_collection(
    collection_name="documents",
    dimension=1536,
    metric_type="COSINE",
    index_params=index_params,
)

client.insert(
    collection_name="documents",
    data=[
        {"id": i, "vector": vec, "text": f"document {i}", "category": "tech"}
        for i, vec in enumerate(embeddings)
    ],
)

results = client.search(
    collection_name="documents",
    data=[query_vector],
    limit=10,
    filter='category == "tech"',
    output_fields=["text", "category"],
)

Metadata Filtering Across Engines

Every engine filters, but the syntax differs enough that abstraction leaks. A quick map:

Engine	Filter example
Pinecone	{"category": {"$eq": "electronics"}, "price": {"$lte": 50}}
Weaviate	Filter.by_property("price").less_than(50.0)
Qdrant	FieldCondition(key="price", range=Range(gte=10, lte=50))
pgvector	WHERE metadata->>'category' = 'electronics'
Chroma	where={"category": {"$eq": "electronics"}}

Push filters into the query whenever you can — post-filtering the top-k gives you fewer than k good results and degrades silently as your corpus grows.

Choosing a Vector Database

The honest answer is that the “best” engine depends more on your constraints than on benchmark charts. Work through these in order:

Constraint	Good default
Under ~1M vectors, already on Postgres	pgvector — no new infrastructure, SQL filters for free
Prototype or local development	Chroma — embedded, zero ops, auto-embeds
No ops team, want managed	Pinecone, Weaviate Cloud, Qdrant Cloud, Zilliz
Billions of vectors or GPU indexing	Milvus, Qdrant
Vector + keyword in one query	Weaviate, Qdrant, Pinecone
Zero budget, self-hosted	Weaviate, Milvus, Qdrant, pgvector

Two follow-up questions settle most remaining ties. First, does your team want to run infrastructure? Self-hosting a vector database means owning index rebuilds, memory pressure, and version upgrades — managed services turn that into an invoice. Second, is the data already in Postgres? pgvector on an existing database removes an entire system from the architecture; it’s the right default until latency, recall, or scale proves otherwise.

RAG Pipeline Example

Chroma is the quickest way to prototype retrieval end-to-end. A runnable version with a Postgres/pgvector backend lives in the companion repository:

from openai import OpenAI
import chromadb

openai_client = OpenAI()
client = chromadb.Client()

# 1. Load and chunk documents
chunks = load_and_chunk_documents("knowledge_base/")

# 2. Embed and store in Chroma
collection = client.create_collection("docs")
resp = openai_client.embeddings.create(input=chunks, model="text-embedding-3-small")
collection.add(
    ids=[f"chunk-{i}" for i in range(len(chunks))],
    documents=chunks,
    embeddings=[e.embedding for e in resp.data],
)

# 3. Retrieve relevant chunks for a query
question = "How do vector indexes work?"
q_resp = openai_client.embeddings.create(input=question, model="text-embedding-3-small")
results = collection.query(query_embeddings=[q_resp.data[0].embedding], n_results=5)

# 4. Augment the LLM prompt with retrieved context
context = "\n".join(results["documents"][0])
prompt = f"Context:\n{context}\n\nQuestion: {question}"
answer = openai_client.chat.completions.create(
    model="gpt-4o", messages=[{"role": "user", "content": prompt}]
)

For the production version of this pattern (batching, retries, source citation), see the RAG Pipeline recipe and the RAG in Production guide.

ANN Algorithms

Algorithm	Type	Speed	Memory	Best For
HNSW	Graph-based	Fast	High	General purpose, high recall
IVF	Clustering	Medium	Medium	Large datasets, memory-constrained
PQ	Quantization	Fast	Low	Billions of vectors, acceptable recall loss

HNSW is the right default in nearly every case. Reach for IVF when the HNSW graph won’t fit in memory, and PQ only when you’re compressing billions of vectors and can tolerate the recall hit.

Benchmarking Vector Search

Never tune an index against vibes. Benchmark two things — recall against a brute-force ground truth, and latency percentiles, not averages:

import time

def benchmark_vector_search(search_fn, query_vectors, ground_truth, top_k=10):
    """ground_truth: list of relevant-id sets, one set per query vector."""
    latencies = []
    hits = 0
    for query, expected_ids in zip(query_vectors, ground_truth):
        start = time.perf_counter()
        results = search_fn(query, top_k)
        latencies.append((time.perf_counter() - start) * 1000)

        result_ids = {r["id"] for r in results[:top_k]}
        if result_ids & expected_ids:
            hits += 1

    latencies.sort()
    n = len(latencies)
    return {
        "recall@k": hits / len(query_vectors),
        "avg_latency_ms": sum(latencies) / n,
        "p50_latency_ms": latencies[n // 2],
        "p95_latency_ms": latencies[int(n * 0.95)],
    }

Get ground truth by running exact k-NN over a representative slice of your corpus — brute force over 10–50K vectors is cheap and gives you the “perfect” answers to score against. Then sweep ef_search (or your engine’s equivalent): you’ll find the knee where recall plateaus and latency keeps climbing. p95 matters more than the average because your users feel the tail.

Common Mistakes

• Wrong distance metric — cosine for semantic text, Euclidean for image features, dot product for normalized embeddings; the wrong metric silently degrades recall
• No metadata filtering — pure vector search returns semantically close but contextually wrong results; always filter by tenant, date, or category
• Leaving index defaults untouched — m and ef_construction at build time, ef_search at query time; defaults are a starting point, not a tuning
• Storing vectors without an index — brute-force scan is O(n) per query and falls over at scale
• Using a vector DB for structured queries — aggregations and joins belong in your relational database

Troubleshooting

• Recall dropped after a bulk insert — the HNSW graph needs rebuilding after large backfills; raise ef_construction and m at build time, and bump ef_search at query time if results look off
• Queries run a full sequential scan — the planner is skipping the index: check EXPLAIN, confirm the table is large enough for index use, and set ivfflat.probes or hnsw.ef_search explicitly
• expected 1536 dimensions, not 768 — the column’s vector(n) doesn’t match the embedding model; vectors from different models can’t share a column, so migrate to a new column sized for the new model
• Search is fast but results are irrelevant — usually a filtering problem, not an index problem: push metadata filters into the query instead of post-filtering the top-k
• Answers degraded after a model upgrade — embeddings from different models aren’t comparable; re-embed everything into a new column and switch queries over (see versioning below)

Advanced Topics

Detailed Scenario: RAG over 50K Documentation Pages

System: RAG chatbot over 50,000 pages of technical documentation
Stack: OpenAI text-embedding-3-small (1536 dim) + pgvector + GPT-4o
Requirements: Accurate answers with citations, latency < 3s

Pipeline:
  1. Load documents (PDF, Markdown, HTML)
  2. Chunk into ~500 token segments with 50 token overlap
  3. Generate embeddings with OpenAI
  4. Store in PostgreSQL with pgvector
  5. On query: embed question, search similar chunks, augment LLM

Schema with a tuned HNSW index:

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE doc_chunks (
    id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    doc_id VARCHAR(100) NOT NULL,
    chunk_index INT NOT NULL,
    content TEXT NOT NULL,
    embedding vector(1536) NOT NULL,
    metadata JSONB DEFAULT '{}',
    created_at TIMESTAMPTZ DEFAULT NOW()
);

CREATE INDEX idx_doc_chunks_embedding
ON doc_chunks USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 64);

CREATE INDEX idx_doc_chunks_doc ON doc_chunks(doc_id);

Ingestion: register_vector makes psycopg2 serialize embeddings correctly:

from openai import OpenAI
import psycopg2
from pgvector.psycopg2 import register_vector

client = OpenAI()
conn = psycopg2.connect("dbname=ragdb")
register_vector(conn)

def embed_text(text: str) -> list[float]:
    resp = client.embeddings.create(input=text, model="text-embedding-3-small")
    return resp.data[0].embedding

def ingest_document(doc_id: str, chunks: list[str]):
    for i, chunk in enumerate(chunks):
        emb = embed_text(chunk)
        with conn.cursor() as cur:
            cur.execute(
                "INSERT INTO doc_chunks (doc_id, chunk_index, content, embedding) "
                "VALUES (%s, %s, %s, %s)",
                (doc_id, i, chunk, emb),
            )
    conn.commit()

Query with source citation:

def rag_query(question: str, top_k: int = 5) -> str:
    q_emb = embed_text(question)
    with conn.cursor() as cur:
        cur.execute(
            "SELECT content, doc_id, chunk_index, "
            "1 - (embedding <=> %s) AS similarity "
            "FROM doc_chunks "
            "ORDER BY embedding <=> %s "
            "LIMIT %s",
            (q_emb, q_emb, top_k),
        )
        results = cur.fetchall()

    context = "\n\n".join(r[0] for r in results)
    sources = [f"doc:{r[1]} chunk:{r[2]} sim:{r[3]:.3f}" for r in results]
    prompt = (
        f"Context:\n{context}\n\n"
        f"Question: {question}\n"
        f"Answer based on the context. Cite the source."
    )
    response = client.chat.completions.create(
        model="gpt-4o", messages=[{"role": "user", "content": prompt}]
    )
    return response.choices[0].message.content + "\n\nSources: " + ", ".join(sources)

Observed metrics on this setup:

Metric	Value
Embedding time	200ms
Vector search time	15ms
LLM time	1.5s
Total latency	~1.7s
Recall@5	92%

HNSW tuning notes: m=16 balances recall against memory, ef_construction=64 controls build quality, ef_search=40 trades recall for latency at query time. At 50K vectors the index holds around 50MB and searches land under 20ms. Chunk size matters more than people expect: 500 tokens with a 50-token overlap outperformed both smaller and larger splits, and filtering by metadata before the vector comparison improved relevance measurably.

Re-embedding When You Change Models

Vectors produced by different models live in different spaces. You can’t query old embeddings with a new model. Store the model name and version in each row’s metadata. When you upgrade, re-embed every document into a new column or table, keep both versions during the transition, switch queries over once the new index is ready, and drop the old column after traffic stops hitting it. Schedule re-embedding in off-peak batches so latency doesn’t spike.

Production Notes

• Size memory for the index: HNSW keeps its graph in RAM — budget roughly 1.5× your raw vector data, or move to IVF/PQ
• Rebuild after bulk loads: large backfills degrade index quality; schedule rebuilds off-peak
• Monitor recall, not just uptime: track p95 query latency and spot-check top-k relevance after index changes
• Back up source documents, not just vectors: you can always re-embed; embeddings you can’t reproduce are gone

Key Takeaways

• pgvector covers most teams under ~1M vectors — dedicated databases earn their cost at scale
• HNSW is the default index; tune m/ef_construction at build and ef_search at query
• Vector similarity alone isn’t enough — always combine it with metadata filtering
• Embeddings are model-specific: version them and re-embed when you change models

Further Reading

• pgvector — extension source, index options, and tuning parameters
• Pinecone documentation — managed vector search concepts and indexes
• Weaviate documentation — hybrid search and schema design
• Chroma documentation — embedded vector store for prototyping
• Qdrant documentation — filtering, indexing, and the query_points API
• Milvus documentation — scaling, index types, and IndexParams
• OpenAI embeddings guide — models, dimensions, and best practices