Vector Embeddings: Models, Search, RAG, and Migration

Vector embeddings are learned numerical representations used to compare text, images, audio, entities, or other data. They power semantic search, recommendations, clustering, and RAG, but a vector alone is not a durable interface. Its meaning depends on the model, revision, task mode, preprocessing, dimensions, normalization, and distance metric that produced it.

This guide owns the engineering lifecycle: model selection, retrieval evaluation, indexing, security, and migration. The glossary owns the concise definition; the semantic search guide owns search-system design; and the vector database guide owns storage-engine selection.

Direct Answer

A production embedding system is a versioned transformation and retrieval contract:

Two vectors are comparable only when they belong to a compatible embedding space. A similarity score is not a probability, a universal relevance threshold, or proof that the source is authorized. Treat the complete contract as data, not as an undocumented SDK setting.

How Vector Embeddings Work

An embedding model learns a function that maps an input into a fixed-length vector so that relationships useful to its training objective become measurable in that space. The model does not preserve every property of the source. It compresses selected signals, which means relevance, language coverage, domain behavior, and failure modes depend on training and task design.

Static, contextual, and retrieval embeddings

A base BERT hidden state is not automatically a good sentence-retrieval embedding. Sentence-BERT introduced a bi-encoder training structure that creates independently computable sentence vectors for similarity search. Modern retrieval models extend this idea with task instructions, asymmetric query/document encoders, multilingual training, or multiple representations.

Symmetric vs. asymmetric tasks

Similarity and retrieval are different objectives. In a symmetric task, both inputs play the same role, such as comparing two paraphrases. In asymmetric retrieval, a short question must find a longer passage that answers it.

Google's embedding documentation makes this distinction explicit with separate task types such as RETRIEVAL_QUERY, RETRIEVAL_DOCUMENT, and SEMANTIC_SIMILARITY. Other models express the same contract through prompts, prefixes, or dedicated encode_query and encode_document methods. Apply the model's documented convention consistently at indexing and query time.

Similarity Metrics Are Part of the Contract

Cosine similarity, dot product, and Euclidean distance answer related but not identical questions unless normalization creates an equivalence.

For vectors x and y:

cosine_similarity(x, y) = (x · y) / (||x|| ||y||)
dot_product(x, y)       = x · y
euclidean_distance(x,y) = ||x - y||₂

When both nonzero vectors are L2-normalized, cosine similarity equals their dot product, and squared Euclidean distance is 2 - 2 * cosine_similarity. They therefore produce the same ordering in exact arithmetic. Without normalization, vector magnitude can change dot-product and Euclidean rankings.

Use the metric recommended by the model and configure the index with the matching operator. OpenAI's current guide uses cosine similarity for text search and requires normalization after manual dimension truncation; this does not establish a rule for every model. Record normalization explicitly even when a provider applies it by default.

Do not copy a similarity cutoff from another dataset. Score distributions move when the model, language, chunking, corpus, task, or negative examples change. Calibrate a decision threshold from labeled examples, and retain a fallback for ambiguous scores.

The Embedding Contract

Persist enough metadata to reproduce, compare, migrate, and delete every vector:

{
  "sourceId": "policy-42",
  "sourceVersion": "sha256:...",
  "chunkId": "policy-42#section-7",
  "tenantId": "tenant-acme",
  "embeddingContract": {
    "modelId": "provider/model-name",
    "modelRevision": "immutable-revision",
    "task": "retrieval_document",
    "preprocessingVersion": "chunk-v4",
    "dimensions": 768,
    "normalization": "l2",
    "distance": "cosine"
  },
  "indexedAt": "2026-01-15T09:30:00Z",
  "accessPolicyVersion": "acl-v12"
}

At minimum, the contract needs:

• Immutable model revision, not only a mutable alias
• Query/document task mode or instruction
• Tokenization, cleaning, chunking, and truncation version
• Output dimension and dtype
• Normalization and distance metric
• Source content hash and lifecycle state
• Tenant, owner, and access-policy reference
• Index generation and ingestion timestamp

This metadata is not optional bookkeeping. Without it, a re-embedding job cannot determine which vectors are stale, a query service can silently use the wrong model, and a deletion request cannot prove that all derived records were removed.

How to Choose an Embedding Model

There is no universally best embedding model. The original MTEB paper evaluated embeddings across retrieval, reranking, semantic textual similarity, classification, clustering, pair classification, bitext mining, and summarization, and found that no single method dominated every task.

Use public benchmarks as a candidate filter, then test the workload you actually operate.

Avoid a model leaderboard copied into permanent documentation. Rankings, model availability, prices, and hardware behavior change. A durable selection record contains the evaluated model revision, dataset version, metrics, operating measurements, and decision.

Build a Retrieval Evaluation Set

An embedding evaluation must represent user retrieval tasks, not only sentence similarity. Start with real or safely synthesized queries and label which documents are relevant.

Include:

1. Direct vocabulary matches and paraphrases
2. Acronyms, aliases, misspellings, and rare entities
3. Questions whose answers do not repeat the query wording
4. Hard negatives that share keywords but answer a different question
5. Multilingual and code-switching queries when supported
6. Permission-restricted documents and expected exclusions
7. No-answer cases where returning nothing is correct
8. Fresh, updated, and deleted source versions

Keep a development set for model and index tuning and a held-out test set for the final comparison. Prevent near-duplicate documents from leaking across those sets.

Retrieval metrics

No single metric is enough:

• Recall@k: whether at least one relevant item appears in the candidate set; useful before reranking.
• MRR: how early the first relevant result appears.
• nDCG@k: rewards useful ordering when relevance has multiple grades.
• Precision@k: how much of the returned set is relevant.
• No-answer precision: whether the system abstains when the corpus lacks an answer.
• Task outcome: whether the downstream search, recommendation, or RAG answer succeeds with valid evidence.

Measure encoding latency, retrieval latency, index build time, memory, storage, and cost separately from relevance. A faster model that misses the required evidence is not an optimization.

A Runnable Local Evaluation

The following Sentence Transformers 5.x example evaluates a real retrieval behavior instead of printing arbitrary vector values. Install with pip install "sentence-transformers>=5,<6" "numpy>=2,<3".

from __future__ import annotations

from dataclasses import dataclass

import numpy as np
from sentence_transformers import SentenceTransformer


MODEL_ID = "sentence-transformers/multi-qa-MiniLM-L6-cos-v1"
# Pin an approved model revision in production.


@dataclass(frozen=True)
class Document:
    id: str
    text: str


DOCUMENTS = [
    Document("refund", "Refund requests are accepted within 30 days of purchase."),
    Document("password", "Reset a password from the account security settings."),
    Document("shipping", "International shipping normally requires customs processing."),
    Document("invoice", "Download tax invoices from the billing history page."),
]

QUERIES = {
    "q1": ("How long do I have to return an order?", {"refund"}),
    "q2": ("Where can I get a receipt for tax?", {"invoice"}),
    "q3": ("I cannot sign in because I forgot my password.", {"password"}),
}


def evaluate_recall_at_k(k: int = 2) -> float:
    if k < 1 or k > len(DOCUMENTS):
        raise ValueError("k must be between 1 and the number of documents")

    model = SentenceTransformer(MODEL_ID)
    document_vectors = np.asarray(
        model.encode(
            [document.text for document in DOCUMENTS],
            normalize_embeddings=True,
            show_progress_bar=False,
        )
    )

    hits = 0
    for query_id, (query, relevant_ids) in QUERIES.items():
        query_vector = np.asarray(
            model.encode(query, normalize_embeddings=True)
        )
        scores = document_vectors @ query_vector
        top_indices = np.argsort(-scores)[:k]
        retrieved_ids = [DOCUMENTS[index].id for index in top_indices]
        hit = bool(relevant_ids.intersection(retrieved_ids))
        hits += int(hit)
        print(f"{query_id}: top{k}={retrieved_ids}, hit={hit}")

    return hits / len(QUERIES)


try:
    recall = evaluate_recall_at_k(k=2)
    print(f"Recall@2={recall:.3f}")
except Exception as error:
    raise RuntimeError(f"Embedding evaluation failed: {error}") from error

This fixture is intentionally small. A production evaluation should load versioned query-document judgments, record the model revision, and compare candidate systems with confidence intervals or per-query regression analysis. Do not tune a model on the same cases used to approve release.

Retrieval Architecture: Exact, ANN, Hybrid, and Reranking

Embeddings are one retrieval signal, not the entire search system.

Sentence Transformers documents the common retrieve-and-rerank pattern: a bi-encoder or lexical system creates a candidate set, then a CrossEncoder scores those candidates jointly with the query. The candidate stage must have sufficient Recall@k because a reranker cannot recover a relevant document that was never retrieved.

Google BigQuery and pgvector both expose exact and approximate search paths, which demonstrates why a dedicated vector database is not mandatory. Start with the simplest store that meets filtering, consistency, scale, recovery, and operational requirements.

Permission-aware pgvector schema

This example stores the embedding contract beside the vector. The fixed vector(384) column belongs to this index generation; a model with a different dimension should use a separate generation or table.

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE document_embeddings_v1 (
  tenant_id text NOT NULL,
  document_id text NOT NULL,
  chunk_id text NOT NULL,
  source_version text NOT NULL,
  model_id text NOT NULL,
  model_revision text NOT NULL,
  preprocessing_version text NOT NULL,
  access_groups text[] NOT NULL,
  embedding vector(384) NOT NULL,
  PRIMARY KEY (
    tenant_id,
    document_id,
    chunk_id,
    model_id,
    model_revision,
    preprocessing_version
  )
);

CREATE INDEX document_embeddings_v1_tenant
  ON document_embeddings_v1 (tenant_id);

CREATE INDEX document_embeddings_v1_access_groups
  ON document_embeddings_v1 USING gin (access_groups);

CREATE INDEX document_embeddings_v1_embedding_hnsw
  ON document_embeddings_v1
  USING hnsw (embedding vector_cosine_ops);

SELECT document_id, chunk_id, source_version,
       1 - (embedding <=> $3::vector) AS cosine_similarity
FROM document_embeddings_v1
WHERE tenant_id = $1
  AND access_groups && $2::text[]
ORDER BY embedding <=> $3::vector
LIMIT $4;

The predicate is illustrative, not a complete authorization boundary. Enforce trusted tenant identity, database row-level security or an equivalent service policy, and do not accept access groups directly from an untrusted client.

With approximate indexes, filtering may happen after part of the ANN scan and reduce the number of eligible results. Measure recall with realistic tenant and metadata filters. Depending on cardinality, use partitioning, partial indexes, iterative scans, or a larger candidate set before reranking.

Dimension and Storage Planning

Higher dimension is not automatically higher quality. Dimension reflects a model's training and output contract; comparing raw dimension counts across unrelated models is not a valid quality test.

A base storage estimate for dense float vectors is:

vector_bytes = item_count * dimensions * bytes_per_element

The complete budget also includes identifiers, metadata, index graph or centroid structures, allocator overhead, replicas, snapshots, write-ahead logs, and old generations retained during migration.

Use a model's native shortening feature only when the model was trained to support it. OpenAI's current third-generation embedding models, for example, expose a dimensions parameter based on Matryoshka-style training. That does not mean arbitrary truncation works for other models.

Post-hoc PCA is a separate learned transformation. It requires a representative fitting dataset, a persisted projection version, normalization rules, evaluation, and a full reindex. The old example on this page attempted to fit 256 PCA components from only six samples, which cannot run because PCA components cannot exceed the available sample rank.

Quantization can reduce storage and index memory, but it can also change neighbor ordering. Evaluate the quantized candidate stage and any full-precision reranking together.

Safe Embedding Migration

Never mix vectors from unrelated embedding contracts in one search space. A safe migration uses parallel generations:

1. Freeze the old contract: record model revision, task mode, preprocessing, dimension, normalization, and metric.
2. Define the new contract: state the hypothesis and approval metrics.
3. Backfill a separate index: preserve source IDs, versions, access policy, and deletion state.
4. Dual-encode queries: send each query to the model that matches each index.
5. Shadow compare: measure quality, latency, failures, and top-result changes without affecting users.
6. Canary traffic: route a bounded cohort with rollback available.
7. Switch reads and writes: keep the old index immutable for the approved rollback window.
8. Retire safely: propagate deletions, remove old vectors and snapshots, and record completion.

An in-place overwrite removes the ability to compare results or roll back. A background job that uses a mutable model alias can also produce multiple vector spaces inside the supposedly new index.

Security and Data Lifecycle

Embeddings are derived data, not anonymized data. OWASP identifies unauthorized access, cross-context leakage, embedding inversion, and data poisoning as vector and embedding risks in RAG systems.

Apply controls before and after vectorization:

• Authenticate sources and reject hidden or untrusted ingestion content.
• Preserve tenant, owner, classification, and provenance with every chunk.
• Enforce authorization in retrieval, not only after text is returned.
• Separate incompatible tenants where shared ANN behavior creates unacceptable leakage or recall effects.
• Encrypt transport and storage; restrict raw-vector export and bulk nearest-neighbor access.
• Propagate source updates, legal holds, expiration, and deletion to vectors, indexes, replicas, caches, and evaluation fixtures.
• Log retrieval identifiers and policy decisions without copying sensitive source text into telemetry.
• Test poisoning, cross-tenant queries, stale ACLs, deleted content, and prompt injection in retrieved passages.

Similarity does not grant permission. A highly relevant private document must remain absent from an unauthorized candidate set.

Common Failure Modes

Production Monitoring

Monitor quality, operations, and lifecycle independently:

• Recall@k, MRR, nDCG@k, no-answer precision, and reranker uplift
• Downstream grounded-answer, click, conversion, or resolution outcome
• Encode and retrieval P50/P95 latency by model and index generation
• Candidate count before and after authorization filters
• Index coverage, stale-vector lag, failed embeddings, and deletion backlog
• Query and document language/domain drift
• ANN recall against a sampled exact-search reference
• Storage, memory, throughput, provider usage, and re-embedding cost

Alert on contract mismatch, not only service failure. A healthy HTTP response from the wrong model revision is a data-corruption event.

Sources and Further Reading

• Vector embeddings, OpenAI API: current API shape, normalization guidance, and model-supported dimension shortening; provider-specific behavior can change.
• Choose an embeddings task type, Google Cloud: symmetric and asymmetric task contracts; task names are specific to Google models.
• Retrieve & Re-Rank, Sentence Transformers: bi-encoder candidate retrieval followed by CrossEncoder reranking.
• MTEB: Massive Text Embedding Benchmark, ACL Anthology: multi-task evidence that model quality does not collapse to one universal ranking.
• Sentence-BERT: the bi-encoder architecture for independently computable sentence representations.
• pgvector: exact and approximate search, distance operators, filtering, and index trade-offs in PostgreSQL.
• LLM08:2025 Vector and Embedding Weaknesses, OWASP: access control, leakage, inversion, and poisoning risks.

Summary

Vector embeddings are useful only inside a controlled, evaluated retrieval system. Choose a model from real task evidence, preserve its full vector-space contract, measure exact and approximate recall, combine lexical retrieval and reranking when the workload needs them, and treat authorization and deletion as part of indexing.

The durable unit is not a floating-point array. It is a versioned relationship between source data, model behavior, retrieval policy, evidence, and lifecycle.

Related Resources

• Semantic Search Engineering Guide - Design lexical, vector, hybrid, and reranked search.
• Vector Database Guide - Compare storage and index responsibilities.
• RAG Retrieval-Augmented Generation Guide - Connect retrieval evidence to generation.
• Advanced RAG Chunking Strategies - Version the preprocessing stage that produces embedding inputs.
• Dense Retrieval Glossary - Distinguish dense retrieval from lexical and hybrid search.