What is BM25?
BM25 is a family of probabilistic lexical ranking functions that scores a document from query-term evidence, inverse document frequency, saturating term frequency, and document-length normalization.
How It Works
How BM25 Computes a Ranking Signal
For a term t, a widely used form multiplies IDF(t) by tf * (k1 + 1) / (tf + k1 * (1 - b + b * dl / avgdl)). k1 controls how quickly repeated occurrences saturate; b controls how strongly field length changes the term contribution. [Lucene 10.3.1](https://lucene.apache.org/core/10_3_1/core/org/apache/lucene/search/similarities/BM25Similarity.html) uses log(1 + (N - df + 0.5) / (df + 0.5)) for IDF and defaults to k1=1.2, b=0.75. Those are implementation defaults, not universal optimums. Tokenization, stemming, stop-word policy, synonyms, field boosts and overlap-token handling can change the ranking before parameter tuning begins.
What the Score Does Not Mean
A BM25 score is not calibrated probability, confidence, answer support or a value that can be compared across queries. It depends on collection statistics, field boundaries and index state; adding documents can change IDF and average length without changing a candidate's text. A high score can still come from repeated but irrelevant terms, and a zero score may reflect analyzer mismatch rather than absence of evidence. Phrase, proximity, recency, authority and access control require explicit query clauses or later ranking features.
Tune and Evaluate the Whole Retrieval Contract
Start with a frozen corpus, analyzer, field schema and relevance judgments. Compare BM25 against Boolean or simpler lexical baselines, then sweep k1, b, field boosts and candidate depth on training queries while reserving held-out queries for release decisions. Report Recall@K for candidate coverage, MRR or nDCG for ordering, p95 latency, zero-result rate and slices for identifiers, long natural-language questions, languages and permissions. The [BEIR benchmark](https://arxiv.org/abs/2104.08663) found BM25 to be a robust zero-shot baseline across heterogeneous tasks, but no benchmark average proves it is best for a particular corpus.
Key Characteristics
- Uses sparse lexical evidence produced by an analyzer rather than learned dense semantics
- Weights corpus-rare terms more strongly through an implementation-specific IDF function
- Saturates repeated term frequency instead of rewarding repetitions linearly
- Normalizes term contributions against document or field length through the b parameter
- Produces query- and index-specific ranking scores rather than calibrated probabilities
- Supports efficient first-stage retrieval through inverted-index execution and pruning
Common Use Cases
- Retrieving runbooks by exact error codes, configuration keys, versions, and API names
- Establishing a reproducible lexical baseline before adding learned retrieval
- Serving the sparse branch of a hybrid RAG candidate pipeline
- Searching legal, policy, or compliance text where wording itself is evidence
- Diagnosing whether ranking failures come from analysis, candidate depth, or later reranking
Example
Loading code...Frequently Asked Questions
Is a BM25 score a probability that a document is relevant?
No. BM25 is motivated by probabilistic relevance research, but the emitted score is an implementation- and query-specific ranking signal. It is not calibrated probability, and a threshold copied from another query, index, field or engine has no stable meaning.
Should every BM25 deployment use k1=1.2 and b=0.75?
No. Those are common Lucene defaults and useful starting points. The best settings depend on field length, analysis, corpus and relevance judgments. Tune them on representative queries and verify the selected configuration on held-out slices rather than treating defaults as a standard.
Why can BM25 miss a document containing the visible query text?
The indexed and queried terms may differ after tokenization, stemming, stop-word removal, Unicode normalization, field selection or access filtering. Inspect the analyzer output and query plan before concluding that the scoring formula failed.
Can BM25 scores be compared across different queries?
Not reliably. Query terms have different document frequencies, and each query creates a different score distribution. Use scores to order candidates within the same retrieval contract; use labeled data to calibrate any cross-query decision or abstention rule.
When should BM25 be combined with dense retrieval?
Test hybrid retrieval when workloads mix exact identifiers and lexical constraints with paraphrases or conceptual questions. Compare BM25, dense and fused candidates on identical relevance judgments, permissions, depth, latency and cost; hybrid is not automatically better.