Vector Database Guide: RAG, pgvector, Search, and Cost

A vector database stores embeddings and retrieves nearby records for semantic search, recommendations, and RAG. It is useful when similarity retrieval is part of the product contract, but it does not have to be a separate product. Exact search, PostgreSQL with pgvector, or a search engine may be the smaller and safer architecture.

This guide owns the storage-engine decision: data contracts, exact versus approximate search, ANN indexes, filters, tenancy, migration, reliability, benchmarking, and cost. The embedding guide covers model selection and vector-space compatibility. The semantic search guide covers retrieval and ranking design.

Table of Contents

• Key Takeaways
• What a Vector Database Actually Does
• Exact Search, Distance Metrics, and Ground Truth
• HNSW, IVF, and Product Quantization
• Filtering, Authorization, and Multitenancy
• How to Choose the Storage Architecture
• A Runnable Go Baseline
• How to Benchmark Candidates
• Embedding Migration and Data Lifecycle
• Reliability and Total Cost
• Vector Databases in RAG
• Common Failure Modes
• FAQ
• Sources
• Summary

Key Takeaways

• A vector database is more than an ANN index. It must preserve IDs, metadata, authorization, update semantics, durability, and observability around similarity retrieval.
• Start with exact search. It gives you both a viable small-scale solution and the ground truth required to measure ANN recall.
• HNSW, IVF, and product quantization are trade-offs, not size-based prescriptions. Compare them at the same recall target, filter mix, and update rate.
• Product selection starts with architecture: in-process search, pgvector, a search engine, a dedicated database, or a managed service.
• Tenant and ACL filters are correctness boundaries. Never retrieve broadly and rely on the LLM to ignore unauthorized context.
• An embedding model change creates a new vector space. Migrate with versioned generations, backfill, shadow evaluation, controlled cutover, and rollback.

What a Vector Database Actually Does

A vector database answers: "Among the records this caller may access, which items are closest to this query vector under a defined metric?" That wording matters. A production request is not just nearest(query, k); it includes an embedding contract, an authorization scope, freshness requirements, and stable links back to source data.

Vector index versus vector database

Faiss is a similarity-search library, not a complete database. Conversely, PostgreSQL becomes a vector-enabled database through pgvector while retaining SQL, transactions, joins, write-ahead logging, and point-in-time recovery. The useful category is therefore not "specialized versus traditional"; it is the operational contract your application receives.

The record contract

A retrievable vector record should carry enough information to validate and reproduce its meaning:

Do not copy every source field into the vector store. Keep the authoritative document in a system of record and project only the fields needed to retrieve, authorize, rank, and trace results. If the vector store is also the system of record, document that choice explicitly because its backup and consistency requirements become stricter.

Ingestion and query paths

The write path and read path have different failure modes:

The database cannot repair a poor embedding model, a lossy chunking policy, an incorrect ACL, or a reranker that promotes irrelevant evidence. It owns retrieval infrastructure, not end-to-end answer correctness.

Exact Search, Distance Metrics, and Ground Truth

Exact nearest-neighbor search compares the query with every eligible vector and returns the true top-k under the selected metric. Its work grows linearly with the number of eligible vectors, but "linear" does not automatically mean "too slow." Small corpora, narrow partitions, batch queries, or optimized numerical libraries can make exact search the right production choice.

Exact search has a second role: it is the oracle for evaluating ANN. Without exact top-k results on a held-out query set, a claimed "fast" index has no measured recall.

Choose the metric from the embedding contract

For unit-normalized vectors, cosine similarity equals the inner product, and squared L2 distance equals 2 - 2 * inner_product; all three produce equivalent rankings. Without normalization, they can produce different results. The correct default is therefore the metric documented and validated for the embedding model, not "cosine for all text" or "dot product because it is fastest."

Distance values are also not calibrated probabilities. A score of 0.82 has no universal meaning across models, model revisions, query types, or corpora. Tune acceptance thresholds on labeled data and version them with the embedding generation.

HNSW, IVF, and Product Quantization

ANN indexes reduce search work by accepting that some exact neighbors may be missed. They should be compared on a recall-latency-memory frontier rather than with labels such as "fastest" or "best for one million vectors."

HNSW

HNSW builds a hierarchy of proximity graphs. Search enters a sparse upper layer, follows promising edges toward the query, descends through denser layers, and explores candidates in the base layer.

The original HNSW paper reports logarithmic complexity scaling for its design and experiments. That is not a universal worst-case O(log N) guarantee for every data distribution, filter, implementation, or parameter setting.

Common controls are:

• M: graph connectivity. Increasing it can improve navigability and recall while increasing memory and build work.
• ef_construction: build-time candidate width. Increasing it can improve graph quality while slowing construction or insertion.
• ef_search: query-time candidate width. Increasing it usually improves recall while increasing latency and CPU work.

HNSW often gives a strong recall-latency trade-off, supports incremental insertion in common implementations, and consumes meaningful graph memory. Deletion and heavy churn remain implementation-specific: tombstones, vacuuming, compaction, or rebuilds can affect both latency and capacity.

IVF

IVF trains centroids, assigns vectors to inverted lists, and probes only selected lists for a query. The major controls are the number of lists and the number of lists probed.

IVF generally builds faster and uses less index memory than HNSW, but recall depends on representative training data and probe count. Distribution drift can make old centroids a poor partition of new vectors. That is why "IVF for batch data" is a useful hypothesis, not a rule: you still need to test rebuild cadence and post-update recall.

Product quantization

Product quantization splits a vector into subvectors and replaces each subvector with a learned codebook entry. It is lossy compression that can reduce memory and accelerate approximate distance calculations. It is often combined with IVF and followed by reranking against full-precision vectors.

There is no universal "10x" or "100x" compression result. The ratio depends on original precision and dimensions, code length, codebook design, stored residuals, and whether full vectors remain available for reranking. The Faiss research overview traces PQ, IVF, and reranking to their original methods and makes the precision-speed-memory trade-off explicit.

Comparison

The current VIBE benchmark evaluates algorithms on modern embedding datasets, including in-distribution and out-of-distribution queries. Its scope explicitly excludes vector databases and cloud services. Use it to form algorithm hypotheses, not to select a production vendor.

Filtering, Authorization, and Multitenancy

Filtering is part of retrieval correctness, not an optional query optimization. A result that is semantically relevant but unauthorized is a security incident.

Pre-filter, in-filter, and post-filter

The pgvector documentation gives a concrete example: approximate-index filtering is applied after the index scan, so a selective condition can leave too few rows. pgvector 0.8 introduced iterative scans that continue searching until enough results are found or a configured limit is reached. Other engines implement filtering differently, so the same test must be repeated for every candidate.

Authorization rules

1. Derive tenant_id, groups, and document permissions from authenticated server-side identity.
2. Apply the authorization predicate inside the retrieval query or before the candidate set leaves the storage boundary.
3. Include the complete authorization scope in every retrieval cache key.
4. Test denied, deleted, cross-tenant, stale-index, retry, and failover cases.
5. Keep audit evidence linking returned chunks to source record, source version, and policy decision.

Application-provided metadata is not trustworthy authorization input. A client must not be able to change tenant_id in a request and broaden its search.

For a small number of large tenants, separate partitions or indexes may improve isolation and predictability. For many small tenants, shared indexes with mandatory filters may be more efficient. Benchmark both the median tenant and the largest or most selective tenants; averages hide noisy-neighbor and recall failures.

How to Choose the Storage Architecture

Choose the smallest architecture that satisfies measured requirements. Corpus size alone is insufficient because vector dimensions, filters, query mix, concurrency, write rate, hardware, and recall target all change the result.

This is a proof-of-work decision:

1. Write the query and data contract.
2. Measure exact search on the eligible subset.
3. Add the least complex indexed option.
4. Load realistic data and filters.
5. Evaluate quality, latency, writes, recovery, and cost.
6. Add a second system only when the evidence identifies a specific limit.

Avoid universal statements such as "pgvector is enough below ten million vectors" or "a dedicated database is required above one million." Those thresholds omit the variables that determine whether a workload succeeds.

A Runnable Go Baseline

This dependency-free Go program implements authorized exact cosine search and Recall@K. It intentionally does not call a database: its role is to produce a deterministic ground-truth contract that any pgvector, search-engine, or dedicated-service adapter must match.

package main

import (
	"errors"
	"fmt"
	"math"
	"sort"
)

type Document struct {
	ID               string
	TenantID         string
	Text             string
	EmbeddingVersion string
	Embedding        []float64
	AllowedUsers     map[string]bool
	Deleted          bool
}

type Result struct {
	ID    string
	Text  string
	Score float64
}

func cosineSimilarity(a, b []float64) (float64, error) {
	if len(a) == 0 || len(a) != len(b) {
		return 0, errors.New("vectors must have the same non-zero dimension")
	}

	var dot, normA, normB float64
	for i := range a {
		dot += a[i] * b[i]
		normA += a[i] * a[i]
		normB += b[i] * b[i]
	}
	if normA == 0 || normB == 0 {
		return 0, errors.New("cosine similarity is undefined for a zero vector")
	}

	return dot / (math.Sqrt(normA) * math.Sqrt(normB)), nil
}

func exactSearch(
	documents []Document,
	query []float64,
	tenantID string,
	userID string,
	embeddingVersion string,
	topK int,
) ([]Result, error) {
	if topK <= 0 {
		return nil, errors.New("topK must be positive")
	}

	results := make([]Result, 0, len(documents))
	for _, document := range documents {
		if document.Deleted ||
			document.TenantID != tenantID ||
			document.EmbeddingVersion != embeddingVersion ||
			!document.AllowedUsers[userID] {
			continue
		}

		score, err := cosineSimilarity(query, document.Embedding)
		if err != nil {
			return nil, fmt.Errorf("score document %s: %w", document.ID, err)
		}
		results = append(results, Result{
			ID:    document.ID,
			Text:  document.Text,
			Score: score,
		})
	}

	sort.Slice(results, func(i, j int) bool {
		if results[i].Score == results[j].Score {
			return results[i].ID < results[j].ID
		}
		return results[i].Score > results[j].Score
	})
	if len(results) > topK {
		results = results[:topK]
	}

	return results, nil
}

func recallAtK(exactIDs, approximateIDs []string) float64 {
	if len(exactIDs) == 0 {
		return 1
	}

	found := make(map[string]bool, len(approximateIDs))
	for _, id := range approximateIDs {
		found[id] = true
	}

	hits := 0
	for _, id := range exactIDs {
		if found[id] {
			hits++
		}
	}
	return float64(hits) / float64(len(exactIDs))
}

func main() {
	documents := []Document{
		{
			ID:               "access-control",
			TenantID:         "acme",
			Text:             "Derive ACL filters from trusted identity.",
			EmbeddingVersion: "embed-v2",
			Embedding:        []float64{0.82, 0.57, 0.05},
			AllowedUsers:     map[string]bool{"alice": true},
		},
		{
			ID:               "retrieval-evaluation",
			TenantID:         "acme",
			Text:             "Compare ANN results with an exact-search baseline.",
			EmbeddingVersion: "embed-v2",
			Embedding:        []float64{0.70, 0.68, 0.10},
			AllowedUsers:     map[string]bool{"alice": true},
		},
		{
			ID:               "private-runbook",
			TenantID:         "acme",
			Text:             "Internal incident response runbook.",
			EmbeddingVersion: "embed-v2",
			Embedding:        []float64{0.84, 0.54, 0.02},
			AllowedUsers:     map[string]bool{"bob": true},
		},
		{
			ID:               "other-tenant",
			TenantID:         "globex",
			Text:             "A document from another tenant.",
			EmbeddingVersion: "embed-v2",
			Embedding:        []float64{0.88, 0.47, 0.01},
			AllowedUsers:     map[string]bool{"alice": true},
		},
		{
			ID:               "old-embedding",
			TenantID:         "acme",
			Text:             "A vector from the previous embedding generation.",
			EmbeddingVersion: "embed-v1",
			Embedding:        []float64{0.90, 0.43, 0.01},
			AllowedUsers:     map[string]bool{"alice": true},
		},
	}

	results, err := exactSearch(
		documents,
		[]float64{0.80, 0.60, 0},
		"acme",
		"alice",
		"embed-v2",
		2,
	)
	if err != nil {
		panic(err)
	}

	for rank, result := range results {
		fmt.Printf("%d. %s score=%.4f\n", rank+1, result.ID, result.Score)
	}

	exactIDs := []string{"access-control", "retrieval-evaluation"}
	approximateIDs := []string{"access-control", "unrelated"}
	fmt.Printf("Recall@2: %.2f\n", recallAtK(exactIDs, approximateIDs))
}

Run it with:

go run main.go

Expected output:

1. access-control score=0.9981
2. retrieval-evaluation score=0.9867
Recall@2: 0.50

Notice what is excluded before scoring: another user's document, another tenant's document, and a vector from an incompatible embedding generation. A production ANN query must preserve those semantics. The Recall@2 example then shows that one missing exact neighbor produces recall of 0.50, even if the ANN request itself returned quickly.

How to Benchmark Candidates

A trustworthy benchmark compares complete workload envelopes, not isolated QPS claims. The ANN-Benchmarks paper established the value of common datasets, parameter sweeps, quality metrics, and reproducible integrations. VIBE extends that idea to modern embedding datasets and out-of-distribution queries.

Freeze the experiment

Record these inputs before testing:

• Corpus snapshot, eligible record count, dimensions, precision, and vector normalization.
• Embedding model, revision, task mode, preprocessing, and distance metric.
• Query set with frequency weights, hard negatives, language mix, and labeled relevance.
• Filter distribution, including selectivity for typical, largest, and smallest tenants.
• Top-k, candidate count, reranking policy, concurrency, and timeout.
• Hardware, region, replicas, memory limits, storage class, and warm versus cold state.
• Insert, update, and delete rate during the test.

Measure all required outputs

Sweep ANN parameters and plot recall against tail latency. Compare products at the same Recall@K rather than at vendor defaults. Then run the application-level relevance evaluation because exact geometric neighbors are not automatically the best evidence for a user question.

For RAG, the production RAG evaluation guide separates retrieval recall, ranking quality, faithfulness, answer quality, latency, and cost.

Embedding Migration and Data Lifecycle

An embedding upgrade is a data migration because vectors from different model contracts do not share a reliable coordinate system. Overwriting old vectors in place destroys rollback and can silently mix incompatible query and document embeddings.

Use a generation-based migration:

1. Define embedding_generation = v2 with model, revision, task mode, dimensions, normalization, metric, and preprocessing.
2. Create separate storage or an index scope for v2.
3. Backfill from immutable source records with idempotent jobs.
4. Reconcile counts, content hashes, failures, and deletion markers.
5. Shadow queries against v1 and v2; compare quality, latency, and cost on the same labeled set.
6. Route a controlled traffic slice to v2 after offline gates pass.
7. Switch the read alias or generation pointer atomically.
8. Retain v1 for a bounded rollback period, then remove it through the normal deletion process.

Inserts, updates, and deletes

Define externally observable semantics:

• Insert freshness: when is a successful write guaranteed to appear in search?
• Update atomicity: can metadata and vector revisions become visible separately?
• Delete propagation: how long can a deleted or revoked record remain retrievable?
• Retry idempotency: does replaying an upsert create duplicates?
• Index rebuild: are writes captured, paused, or replayed while a new index builds?

Soft deletion may help recovery, but it is not authorization. Queries must exclude deleted records immediately according to the product contract, and background compaction must eventually reclaim their storage and graph edges.

Reliability and Total Cost

A vector database is production-ready only when its data can be reconstructed, restored, and reconciled under failure. Backing up an index alone is insufficient if source records, metadata, model version, or chunking configuration are missing.

Recovery contract

Document and test:

• Which data is authoritative and which indexes are disposable projections.
• Backup coverage for vectors, metadata, aliases, configuration, and source records.
• Recovery point objective and recovery time objective.
• Restore into an isolated environment followed by count, hash, ACL, and sample-query checks.
• Behavior during regional failure, partial ingestion, queue replay, and index rebuild.
• Audit logs for access, mutation, deletion, and migration.

When PostgreSQL with pgvector is the store, its vectors participate in PostgreSQL WAL, replication, and point-in-time recovery. A separate vector service needs its own verified backup and reconciliation path; "managed" does not remove the need to test restore.

Cost model

The raw vector lower bound is:

raw_vector_bytes = vector_count * dimensions * bytes_per_component

That number excludes graph edges or quantized codes, record IDs, metadata indexes, replicas, write buffers, compaction headroom, backups, and network traffic. A realistic monthly comparison includes:

total_cost =
  steady_compute + peak_headroom + primary_storage + index_storage
  + replicas + backups + network_egress + ingestion
  + operations_and_incident_time

Measure cost at the required recall, p99 latency, freshness, and availability. A cheaper configuration that violates a quality or security gate is not an equivalent option.

Vector Databases in RAG

A vector database is the candidate-retrieval layer of RAG, not the complete RAG system.

query
  -> query normalization or rewrite
  -> authorized lexical and vector retrieval
  -> candidate fusion
  -> reranking
  -> context assembly with source identities
  -> generation
  -> citation and answer evaluation

Dense retrieval can miss identifiers, error codes, names, and rare terms. Lexical retrieval can miss paraphrases. Hybrid search combines their candidate sets, while a reranker can spend more computation on a smaller pool. The database should return enough traceable candidates for those later stages rather than pretending a vector similarity score is final answer confidence.

Repeated, authorized requests may benefit from a semantic cache, but cache keys must include tenant, permissions, embedding generation, retrieval configuration, model, prompt, and source versions. A cache hit must never bypass current authorization or deletion state.

Common Failure Modes

FAQ

What is the difference between a vector database and a relational database?

A relational database is a general data system centered on structured records, constraints, transactions, joins, and scalar indexes. A vector database or vector-enabled database adds similarity retrieval over high-dimensional embeddings. The categories overlap: PostgreSQL with pgvector performs vector search, while a dedicated vector engine may offer only limited relational behavior. Choose by required contracts, not category labels.

Does every RAG application need a dedicated vector database?

No. A small corpus may be served by exact search. PostgreSQL plus pgvector is attractive when vectors and business rows need shared transactions and SQL filters. A search engine is attractive when keyword retrieval, facets, and vector candidates share one ranking stack. A dedicated or managed vector system becomes justified when its measured scale, filtering, distribution, or operational model solves a real limit.

Is HNSW always better than IVF?

No. HNSW often has a favorable recall-latency frontier, while IVF can reduce memory and build cost for stable datasets. HNSW graph memory and churn may be unacceptable; IVF centroid training and drift may be unacceptable. Compare them using the same dataset, filters, updates, and Recall@K target.

How should vector database retrieval quality be measured?

First compare ANN results with exact top-k neighbors to calculate Recall@K. Then measure user relevance with labeled queries using Precision@K, MRR, or nDCG. Finally measure RAG answer quality separately. Geometric recall, retrieval relevance, and answer faithfulness are related but not interchangeable.

How do you change embedding models without downtime?

Create a separate embedding generation, backfill it, shadow or dual-read both generations, and compare quality plus latency. Switch an alias only after release gates pass. Keep the previous generation during a rollback window and propagate updates and deletes to both until the migration closes.

Sources

• Efficient and Robust Approximate Nearest Neighbor Search Using HNSW - original HNSW mechanism and evaluated scaling.
• Faiss Documentation - exact and approximate search, IVF, PQ, GPU, and evaluation foundations.
• pgvector Documentation - exact search, HNSW, IVFFlat, filtering, iterative scans, updates, replication, and monitoring.
• ANN-Benchmarks Paper - reproducible ANN quality-performance comparison methodology.
• VIBE - modern embedding datasets and algorithm-level benchmark scope.

Summary

A sound vector database choice begins with an authorized query contract and an exact baseline. Select an ANN index only after measuring the recall-latency-memory trade-off, and select a storage architecture only after testing filters, writes, recovery, migration, and total cost. Keep embeddings versioned, enforce permissions inside retrieval, and evaluate RAG answers separately from vector-neighbor accuracy.

Related Resources

• Vector Embeddings Engineering Guide - define model contracts, evaluate embeddings, and migrate generations.
• Semantic Search Engineering Guide - combine lexical, vector, filtered, and reranked retrieval.
• RAG Retrieval-Augmented Generation Guide - build the complete retrieval and generation pipeline.
• Production RAG Evaluation - establish retrieval and answer-quality release gates.