What is Semantic Search?
Semantic Search is information retrieval that uses learned representations to find documents related to a query's meaning, usually as one stage in a pipeline that also applies lexical retrieval, filters, fusion, reranking, and evaluation.
Quick Facts
| Created | Concept from 2000s, transformer-based from 2019 |
|---|---|
| Specification | Official Specification |
How It Works
Retrieval pipeline
Start with an explicit corpus contract: document identity, chunk boundaries, source revision, language, owner, tenant, access policy, and deletion behavior. Encode and index only the fields intended for search. At query time, normalize the request, apply trusted scope constraints, retrieve a broad candidate set, rerank a smaller set, and return source identifiers with the final results. For RAG, retrieval quality and answer quality are separate measurements.
Dense, lexical, and hybrid retrieval
Dense retrieval uses query and document embeddings, often with an approximate nearest-neighbor index, to recall paraphrases and conceptual matches. Lexical methods such as BM25 remain strong for product codes, names, quoted phrases, rare tokens, and code symbols. Hybrid retrieval combines independently produced rankings, commonly with reciprocal rank fusion. The fusion method and weights must be validated on the target query mix rather than assumed to improve every workload.
Filtering and trust boundaries
Similarity is not authorization. Tenant, object, retention, geography, sensitivity, and freshness checks should come from trusted metadata and be enforced before a result reaches the model or user. Do not ask an embedding model to infer permission from document text. Preserve source and revision metadata through chunking and indexing so deletion, correction, citation, and incident investigation can trace a result back to an authoritative record.
Reranking and score meaning
A bi-encoder can score a large corpus efficiently because document vectors are precomputed. A cross-encoder jointly reads a query-document pair and is usually reserved for a shortlist because it costs more per candidate. Cosine, dot-product, lexical, and reranker scores live on different scales; they are ranking signals, not calibrated probabilities or proof of factual support. Thresholds must be fitted and monitored for the chosen model, index, corpus, and query class.
Evaluation and change control
Create labeled queries that cover common intents, long-tail language, exact identifiers, no-answer cases, stale content, and access boundaries. Use Recall@K to test candidate coverage, MRR for the first relevant hit, and nDCG when relevance is graded; also record latency, cost, zero-result rate, policy violations, and downstream answer support. Re-run the same suite whenever the embedding model, chunker, corpus, index parameters, filters, fusion, or reranker changes.
Key Characteristics
- Meaning-aware recall: embeddings can retrieve paraphrases and conceptual matches without exact token overlap
- Complementary retrieval: lexical search remains important for identifiers, names, phrases, and rare terms
- Explicit policy filters: authorization, tenant, freshness, and retention rules do not come from vector similarity
- Two-stage ranking: efficient retrievers generate candidates and more expensive rerankers refine the shortlist
- Index tradeoffs: approximate nearest-neighbor settings balance recall, latency, memory, and update cost
- Measured relevance: model and pipeline choices are validated with labeled queries, error slices, and online evidence
Common Use Cases
- Retrieving sourced and authorized passages for a RAG answer
- Combining natural-language product discovery with inventory and catalog filters
- Searching enterprise knowledge under tenant, role, retention, and freshness constraints
- Routing support cases with a relevance benchmark reviewed by domain experts
- Finding code by intent while preserving exact symbol and identifier search
Example
Loading code...Frequently Asked Questions
Is semantic search the same as vector search?
Not exactly. Vector search is a mechanism for finding nearby vectors, while semantic search is the retrieval goal of matching meaning. Semantic search often uses vector search over text embeddings, but it also needs document design, filters, ranking, and evaluation; vector search can also serve non-text similarity tasks.
When should semantic search be combined with keyword search?
Test a hybrid approach when queries mix natural-language intent with exact entities, identifiers, error codes, quotations, or rare terms. Retrieve lexical and semantic candidates independently, apply trusted filters, fuse their ranks, and compare the result with each single retriever on the same labeled queries.
Does a high cosine similarity mean a document is relevant?
No. The score is relative to the embedding model, normalization, index, and corpus. It is not a calibrated probability, does not prove factual support, and says nothing about authorization or freshness. Fit thresholds from labeled data and inspect errors by query class.
When is a reranker worth the extra latency?
A reranker is useful when first-stage recall is adequate but top positions contain near-misses that require joint query-document reasoning. Measure the quality gain at several candidate depths and include model cost and tail latency; reranking cannot recover a relevant document that the retriever never returned.
How do I evaluate semantic search quality?
Build a versioned relevance set from representative queries and judgments. Use Recall@K for candidate coverage, MRR for the first relevant result, and nDCG for graded ranking, then slice failures by language, intent, identifier use, freshness, permissions, and no-answer cases. Track latency, cost, and downstream answer support separately.