Vector Databases Compared: Pinecone vs Weaviate vs Qdrant vs pgvector

Choosing a vector database feels like a product decision, but most of the outcome is determined by a few engineering facts: how many vectors you have, how many dimensions, how selective your filters are, and how much recall you are willing to trade for latency. Once you understand those, the comparison between Pinecone, Weaviate, Qdrant and pgvector becomes straightforward. This guide starts with how the indexes work, does the memory math, explains why filtering is the hard part, then compares the four with working code and a decision table.

Product features in this space move quickly. Everything below reflects the systems as they are generally documented in 2026; check each project's current docs for limits and pricing before you commit.

What a Vector Database Actually Does

An embedding model turns text, images or code into a vector of numbers, typically a few hundred to a few thousand dimensions, such that similar content ends up close together. If you need a refresher on how embeddings are produced, read embeddings and semantic search. A vector database then answers one core query:

Given a query vector q, return the k stored vectors closest to q under a distance metric (cosine, dot product or Euclidean), optionally restricted to items matching a filter.

Doing this exactly means computing the distance to every stored vector: O(N·d) per query. For 1 million 768-dimension vectors that is about 768 million multiply-adds per query, which a modern CPU with SIMD can do in tens of milliseconds. That is fine for a prototype and too slow and expensive at high QPS or larger N. ANN indexes trade a little recall (the fraction of the true top-k you actually return) for orders-of-magnitude faster search.

Around that core, a database adds the things you need in production: persistence, updates and deletes, metadata storage and filtering, replication, backups, access control and multi-tenancy. That is what separates a vector database from a library like FAISS or hnswlib.

Distance metrics in one minute

• Cosine similarity compares direction only. Most text embedding models are trained for it.
• Dot product equals cosine when vectors are normalised to unit length, and is slightly cheaper. Many models output normalised vectors, in which case the two give the same ranking.
• Euclidean (L2) distance also gives the same ranking as cosine on normalised vectors.

Use the metric the embedding model's documentation recommends. Mismatched metrics are a silent recall bug.

How ANN Indexes Work: HNSW, IVF and Quantization

HNSW: a navigable graph

Hierarchical Navigable Small World graphs are the default index in all four systems. Each vector is a node connected to its nearest neighbours. Upper layers contain exponentially fewer nodes and act as express lanes. A search starts at the top layer, greedily hops to whichever neighbour is closest to the query, drops a layer, and repeats until the bottom layer, where it explores a candidate list to collect the top k.

Three parameters control the trade-off, and the names are nearly identical across engines:

HNSW's strengths are excellent recall-latency trade-offs and support for incremental inserts. Its weaknesses: the graph lives best in RAM, builds are CPU-intensive, and deletes are handled by marking nodes, which degrades the graph until it is cleaned up or rebuilt.

IVF: cluster, then search a few clusters

An inverted file index runs k-means to pick nlist centroids and assigns every vector to its nearest centroid. At query time it finds the nprobe closest centroids and scans only those lists. With 1,000 lists and nprobe 10, you scan about 1% of the data. IVF builds faster and uses less memory than HNSW, but recall is sensitive to nprobe and to whether the data distribution has drifted since clustering. In pgvector this is the ivfflat index, and it should be built after the table has representative data, because the centroids come from the existing rows.

Quantization: storing fewer bytes per vector

• Half precision (float16): 2 bytes per dimension instead of 4. Usually negligible recall loss. pgvector exposes this as the halfvec type.
• Scalar quantization (int8): map each dimension to one byte. 4× smaller, small recall loss, usually recovered by rescoring top candidates with full vectors.
• Binary quantization: one bit per dimension (the sign). 32× smaller, fast Hamming distance, larger recall loss. Works best with high-dimensional models and an oversample-then-rescore step.
• Product quantization (PQ): split the vector into m sub-vectors and replace each with the ID of its nearest sub-centroid. A 1,536-dimension vector stored as 96 one-byte codes is 64× smaller. Common with IVF at billion scale.

The general pattern in modern engines is search on compressed vectors, rescore with originals: fetch, say, 4× k candidates using the compact representation, then compute exact distances on those candidates using full-precision vectors kept on disk.

Memory Math: Do This Before Choosing Anything

The raw size of your vectors is N × d × bytes per value. Here is a worked example for 10 million chunks, which is a realistic size for a mid-size company's document corpus or a product catalogue with multiple embeddings per item.

Add the HNSW graph. The bottom layer typically holds up to 2·M neighbours per node; at M = 16 with 4-byte IDs that is about 128 bytes, plus upper layers and engine overhead, so plan for very roughly 100–200 bytes per vector. For 10 million vectors that is another 1–2 GB, small next to float32 vectors but significant next to binary ones.

Two conclusions follow. First, dimension choice dominates. Several current embedding model families support shortened outputs (Matryoshka-style training), letting you request 512 or 256 dimensions with modest quality loss; measure that loss on your data before paying for 3× the memory. Second, at 10 million float32 vectors, you either buy a large-memory machine, quantize, or use an engine that keeps full vectors on disk and only compressed ones in RAM.

Filtered Search: Where the Engines Really Differ

Real queries are rarely "nearest vectors in the whole corpus". They are "nearest vectors for tenant 42, in documents the user can access, published after 2025". Filtering is hard because the ANN index knows nothing about metadata. There are three basic strategies:

1. Post-filtering. Run the ANN search, then drop results that fail the filter. Simple, but if only 2% of vectors match the filter and you fetched 40 candidates, you get about one result back.
2. Pre-filtering. Compute the set of matching IDs first, then search only those, often by brute force when the set is small. Accurate, but expensive when the filter matches a large fraction.
3. Filter-aware graph traversal. Walk the HNSW graph but only count matching nodes as results, while still traversing through non-matching ones. Needs care so the graph stays connected for restrictive filters.

How the four handle it, as generally documented:

• pgvector historically post-filtered from the index scan, so selective filters could return too few rows (with the default hnsw.ef_search of 40 and a filter matching 10% of rows, you get roughly 4 rows). Version 0.8.0 added iterative index scans (hnsw.iterative_scan), which keep scanning the index until enough rows pass the filter. You can also use partial indexes or table partitioning per tenant, and Postgres may choose an exact scan via a B-tree index when the filter is very selective.
• Qdrant builds payload indexes and uses a query planner that chooses between filtered HNSW traversal and exact search over the filtered set based on estimated cardinality. It also adds extra graph links for indexed payload values so filtered traversal stays connected.
• Weaviate builds an inverted index on properties, computes an allow-list and searches the HNSW graph against it, switching to a flat search when the allow-list is small. It also supports multi-tenancy with a separate index per tenant.
• Pinecone supports metadata filtering inside the query and namespaces to physically partition data, which is the cleanest option for per-tenant isolation.

The practical rule: if a filter is a hard partition (tenant, language, region), make it a partition (namespace, tenant, table partition or separate collection). Use metadata filters for soft, variable conditions such as date ranges or categories.

The Four Options in Depth

pgvector: vectors inside Postgres

An open-source Postgres extension adding vector, halfvec, bit and sparsevec types, distance operators and HNSW and IVFFlat indexes. Available on most managed Postgres services. Its big advantage is not raw speed but architecture: embeddings live next to the rows they describe, updates are transactional, you can join with permissions tables, and you operate one database instead of two. The limits: index sizes are bounded by what a single Postgres node can hold and build (building HNSW for tens of millions of vectors needs substantial maintenance_work_mem and time), indexed vector columns are limited to 2,000 dimensions (halfvec to 4,000), and horizontal scaling means read replicas or sharding extensions such as Citus.

Pinecone: fully managed and serverless

A proprietary managed service. The serverless architecture separates storage from compute, so you pay for storage and for read and write usage rather than for provisioned machines. You get namespaces, metadata filtering, sparse-dense hybrid search and hosted embedding and reranking models. Strengths: nothing to operate, scales to large indexes, good developer experience. Trade-offs: data leaves your infrastructure (check region availability and data residency requirements, which matter for Indian fintech and health data), costs grow with query volume, and you cannot tune internals.

Qdrant: Rust engine with strong filtering

Open source (Apache 2.0) with a managed cloud. Points carry a vector (or several named vectors) and a JSON payload. Notable features: payload indexes and a filter-aware planner, scalar, binary and product quantization with rescoring, on-disk vectors and indexes, sparse vectors, and a query API that combines multiple prefetch stages with fusion for hybrid search. It runs as a single binary or as a distributed cluster with sharding and replication. A good fit when you want to self-host, need heavy filtering, or need to fit large collections in limited RAM.

Weaviate: hybrid search and modules

Open source (BSD-3) with a managed cloud. Objects live in collections with a schema. Its standout features are built-in hybrid search (BM25 and vector fused, with an alpha weight), native multi-tenancy, a choice of HNSW, flat or dynamic indexes, several compression options, and modules that call embedding and generative models for you at import and query time. A good fit when keyword relevance matters alongside semantic search and when you want vectorization handled inside the database.

Others worth knowing

Milvus (distributed, GPU index support, billion-scale), Elasticsearch and OpenSearch (add vectors to an existing search cluster; see the inverted index guide), Redis (low-latency vectors next to a cache), MongoDB Atlas Vector Search, and embedded options such as Chroma and LanceDB for local or prototype work.

The Same Filtered Query in Four APIs

Task: store document chunks with a tenant field, then find the 5 chunks most similar to a query vector for tenant acme. Embedding generation is omitted; assume vec is a list of 1,536 floats.

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE chunks (
  id        bigserial PRIMARY KEY,
  tenant    text NOT NULL,
  content   text NOT NULL,
  embedding halfvec(1536) NOT NULL
);

CREATE INDEX ON chunks USING hnsw (embedding halfvec_cosine_ops)
  WITH (m = 16, ef_construction = 64);
CREATE INDEX ON chunks (tenant);

-- per session or transaction
SET hnsw.ef_search = 100;
SET hnsw.iterative_scan = relaxed_order;   -- pgvector 0.8.0+

SELECT id, content, embedding <=> $1::halfvec AS distance
FROM chunks
WHERE tenant = 'acme'
ORDER BY embedding <=> $1::halfvec
LIMIT 5;

<=> is cosine distance (1 − cosine similarity), so smaller is closer. The ORDER BY distance LIMIT k shape is what lets Postgres use the HNSW index; wrapping the distance in another expression prevents it. With relaxed_order, results can be slightly out of order, so re-sort in an outer query if exact ordering matters.

from qdrant_client import QdrantClient, models

client = QdrantClient(url="http://localhost:6333")
client.create_collection(
    collection_name="chunks",
    vectors_config=models.VectorParams(size=1536, distance=models.Distance.COSINE),
)
client.create_payload_index("chunks", field_name="tenant", field_schema="keyword")

client.upsert("chunks", points=[
    models.PointStruct(id=1, vector=vec, payload={"tenant": "acme", "content": "..."}),
])

hits = client.query_points(
    collection_name="chunks",
    query=vec,
    query_filter=models.Filter(must=[
        models.FieldCondition(key="tenant", match=models.MatchValue(value="acme"))
    ]),
    limit=5,
).points

import weaviate
from weaviate.classes.config import Property, DataType
from weaviate.classes.query import Filter

client = weaviate.connect_to_local()
chunks = client.collections.create(
    "Chunk",
    properties=[Property(name="tenant", data_type=DataType.TEXT),
                Property(name="content", data_type=DataType.TEXT)],
)   # no vectorizer configured: we supply our own vectors
chunks.data.insert({"tenant": "acme", "content": "..."}, vector=vec)

res = chunks.query.near_vector(
    near_vector=vec, limit=5,
    filters=Filter.by_property("tenant").equal("acme"),
)
client.close()

from pinecone import Pinecone, ServerlessSpec

pc = Pinecone(api_key="...")
pc.create_index(name="chunks", dimension=1536, metric="cosine",
                spec=ServerlessSpec(cloud="aws", region="us-east-1"))
index = pc.Index("chunks")

index.upsert(vectors=[{"id": "1", "values": vec, "metadata": {"content": "..."}}],
             namespace="acme")           # tenant as a namespace

res = index.query(vector=vec, top_k=5, namespace="acme", include_metadata=True)

Client APIs evolve (Weaviate's collection configuration helpers in particular have changed between versions), so treat these as shapes and check the current client docs. Notice the design choice in the Pinecone example: the tenant is a namespace, not a filter, because tenant isolation is a hard partition.

Side-by-Side Comparison

Cost Models: How You Actually Pay

Instead of quoting prices that change every few months, here is how each model behaves as you grow.

• pgvector: cost is the Postgres instance. Mostly RAM-driven: HNSW performs best when the index fits in memory. Marginal cost of vectors is low until you need a bigger instance tier. No per-query fees.
• Pinecone serverless: pay for storage plus read and write units consumed. Very cheap for small or spiky workloads; at sustained high QPS, usage-based billing can exceed the cost of a dedicated cluster. Model your expected monthly queries before committing.
• Qdrant and Weaviate self-hosted: pay for machines plus your team's time. With quantization and on-disk storage, RAM per vector can drop by an order of magnitude.
• Qdrant Cloud and Weaviate Cloud: priced by provisioned resources or usage depending on the plan; usually in between the other two.

A useful exercise: compute cost per million queries and cost per million stored vectors per month for each candidate at your projected scale one year out. The cheapest option at 100k vectors is rarely the cheapest at 50 million.

How to Benchmark Vector Databases Yourself

Public benchmarks such as ann-benchmarks (algorithm level) and VectorDBBench are useful for intuition, but some are vendor-authored and none use your data, filters or hardware. A credible internal benchmark takes a day or two.

1. Use your real embeddings. Embed a representative sample (say 1–5 million chunks) with the model you will ship.
2. Build ground truth with exact search. For 1,000 real or realistic queries, compute the true top-10 by brute force.
3. Measure recall@10 against latency. Sweep ef_search (or the engine's equivalent) and plot recall against p50 and p99 latency at your target concurrency.
4. Repeat with filters at the selectivities you expect, for example 50%, 5% and 0.5% of the corpus.
5. Measure ingest and update behaviour: build time, insert throughput while serving queries, and recall after many deletes.
6. Record memory and cost at the chosen operating point.

import numpy as np

def exact_topk(corpus, queries, k=10):
    # corpus: (N, d), queries: (Q, d); both L2-normalised so dot = cosine
    scores = queries @ corpus.T                          # (Q, N)
    idx = np.argpartition(-scores, k, axis=1)[:, :k]     # unordered top-k
    order = np.argsort(-np.take_along_axis(scores, idx, axis=1), axis=1)
    return np.take_along_axis(idx, order, axis=1)

def recall_at_k(truth, found, k=10):
    hits = sum(len(set(t[:k]) & set(f[:k])) for t, f in zip(truth, found))
    return hits / (len(truth) * k)

rng = np.random.default_rng(0)
corpus = rng.standard_normal((20000, 256)).astype(np.float32)
corpus /= np.linalg.norm(corpus, axis=1, keepdims=True)
queries = corpus[:100] + 0.05 * rng.standard_normal((100, 256)).astype(np.float32)
queries /= np.linalg.norm(queries, axis=1, keepdims=True)

truth = exact_topk(corpus, queries)
print(truth[0][0])                  # 0: each query's nearest neighbour is its source
print(recall_at_k(truth, truth))    # 1.0; pass your database's results as 'found'

Recall measures retrieval fidelity against exact search, not answer quality. For end-to-end quality, pair this with retrieval and answer evals, covered in our LLM evals guide. Often a better chunking strategy or a reranker improves answers more than moving recall from 0.95 to 0.99.

Decision Guide: Which Vector Database Should You Use?

My default advice for most Indian startups building their first RAG feature: start with pgvector on the Postgres you already run, using halfvec and HNSW, and put retrieval behind an interface so you can swap engines later. Move only when a benchmark on your data shows a concrete problem. The RAG guide covers the rest of the pipeline, and LangChain vs LlamaIndex vs plain SDKs discusses whether to wrap any of this in a framework.

How Vector Databases Come Up in Interviews

AI engineering and backend interviews increasingly include a retrieval question, either inside a system design ("design a support chatbot over 5 million help articles") or as a standalone discussion. What interviewers probe:

• Do the sizing. "5M chunks × 1,024 dims × 4 bytes ≈ 20 GB raw; with HNSW and headroom I'd want about 32 GB RAM, or I'd use half precision." Candidates who do this immediately stand out.
• Explain HNSW in two sentences and name the knob that trades recall for latency (ef_search).
• Raise filtering and multi-tenancy unprompted. "Each customer must only see its own documents, so I'd partition by tenant rather than post-filter."
• Talk about re-embedding. When you change embedding models, you cannot mix vectors from two models in one index; build a new index side by side, backfill, then cut over.
• Know when not to use one. For 50,000 FAQ entries, exact search in memory is simpler and perfectly accurate.

A red flag interviewers mention: treating the vector database as the whole of RAG. Retrieval quality depends at least as much on chunking, hybrid keyword search, reranking and evaluation as on the index. If you can explain how you would measure recall and answer quality, and why you picked the engine you picked, you are ahead of most candidates. For the end-to-end design, see designing an LLM chat application.

Frequently Asked Questions

Pranjal Jain

Ex-Microsoft SDE · IIT Kanpur · Founder of Prepflix. Helps engineers crack startup and product company interviews across India.