What is Dense Retrieval?
Dense Retrieval is a first-stage search method that encodes queries and documents into compact numeric vectors and ranks candidates by a declared vector similarity or distance function.
How It Works
Define the Embedding Contract
Record separate query and document preprocessing, model revision, tokenizer, task prefix, pooling rule, output dimension, numeric dtype and whether vectors are normalized. Cosine similarity equals dot product only for normalized vectors; Euclidean distance has a different ordering contract. Never mix vectors from different revisions in one logical index unless compatibility has been demonstrated. A migration should build a parallel index and compare overlap, labeled relevance, latency and downstream answer evidence before traffic moves.
Separate Model Recall from ANN Recall
The encoder decides where relevant and irrelevant items land in vector space; the index decides whether search finds the nearest candidates under a resource budget. Evaluate exact search on a representative subset, then measure ANN Recall@K against that exact result while sweeping HNSW, IVF, quantization or equivalent settings. Apply metadata and permission filters with documented pre-filter or post-filter semantics. A high ANN Recall cannot repair a model that embedded the relevant passage far from the query.
Evaluate Retrieval Before Generation
Use query-level relevance judgments and report candidate Recall@K, MRR or nDCG, p50/p95 latency, index memory, update lag and no-result or filter-exhaustion rates. Slice paraphrases, exact IDs, negation, language, time, permissions and out-of-domain queries. The original [DPR study](https://aclanthology.org/2020.emnlp-main.550/) established a practical dual-encoder route for open-domain QA, while [BEIR](https://arxiv.org/abs/2104.08663) later showed that zero-shot generalization differs substantially by task. Treat every published gain as conditional on its corpus, labels and retrieval protocol.
Key Characteristics
- Maps queries and documents into compact vectors in a compatible representation space
- Supports paraphrase and conceptual matching without requiring exact token overlap
- Uses exact or approximate nearest-neighbor search for first-stage candidate generation
- Separates encoder quality from index approximation, filtering and update behavior
- Produces model-specific similarities rather than calibrated relevance probabilities
- Requires coordinated model, tokenizer, preprocessing and index revision management
Common Use Cases
- Retrieving semantically related evidence for RAG and question answering
- Searching support tickets or documentation across paraphrases
- Finding multilingual candidates with a model evaluated for the target language pair
- Generating a high-recall candidate set for later reranking
- Comparing embedding and ANN revisions through shadow indexes before migration
Example
Loading code...Frequently Asked Questions
Are dense retrieval, vector search, and semantic search identical?
They overlap but are not identical. Dense retrieval describes candidate ranking with dense representations. Vector search is an execution primitive that can serve non-semantic vectors, while semantic search is a user intent that may combine dense, sparse and reranking signals.
Should dense retrieval use cosine similarity or dot product?
Use the metric required by the model and training objective. Cosine and dot product rank identically only when vectors are normalized. Changing normalization or metric changes score semantics and requires a new compatibility and relevance evaluation.
Must documents be re-embedded when the embedding model changes?
Usually yes. A model, tokenizer, prefix, pooling or dimension change can create a different vector space. Build a parallel index and validate it; never silently query old document vectors with a new query encoder unless compatibility is explicitly guaranteed.
Why can a high cosine score still return the wrong passage?
Similarity reflects the learned geometry, not answer correctness. Domain shift, chunking, negation, numbers, identifiers and missing permissions can all produce misleading neighbors. Evaluate with labeled queries and downstream evidence checks rather than a universal score threshold.
Can a reranker recover a relevant document missed by dense retrieval?
No. A reranker can only reorder candidates it receives. Candidate Recall@K must be measured before reranking, and lexical or hybrid branches may be needed when dense retrieval systematically misses exact or domain-specific evidence.