What is Context Recall?
Context Recall is a RAG evaluation metric that measures the proportion of a declared required-evidence set represented in the retrieved context, using stable evidence IDs or reference claims as the scoring units.
Quick Facts
| Specification | Official Specification |
|---|
How It Works
Choose an evidence universe
ID-based recall divides unique required evidence IDs retrieved in top-k by all unique required IDs. It is reproducible when annotations and source versions are stable, but it can be too coarse if a whole document is labeled for one passage. Claim-based recall decomposes a trusted reference answer and measures which claims the retrieved context supports. It works without exact passage IDs but inherits claim-extraction and entailment-judge error.
Control granularity, duplicates, and unanswerable queries
A document ID, chunk ID, fact, and reference claim produce different denominators. Deduplicate repeated chunks or aliases before scoring, preserve source-version mappings after reindexing, and label questions whose answer is absent from the corpus. An unanswerable query should exercise abstention behavior; it should not be assigned a fabricated recall denominator.
Interpret recall as a retrieval diagnostic
Low recall may come from missing ingestion, stale indexes, segmentation boundaries, vocabulary mismatch, restrictive metadata filters, weak candidate retrieval, or insufficient K. Raising K can improve coverage while adding noise and cost. Compare changes with context precision, rank position, answer faithfulness, and task success, then inspect misses by query class before choosing query rewriting, hybrid search, or corpus repair.
Key Characteristics
- Requires a declared reference set of evidence IDs or answer claims
- Measures coverage of required evidence rather than cleanliness of returned context
- Changes with evidence granularity, source version, K, and duplicate policy
- Can expose ingestion, filtering, chunking, and candidate-retrieval failures
- Cannot be computed faithfully from retrieved results alone
- Must be paired with precision because broader retrieval can add harmful noise
Common Use Cases
- Checking whether gold passages appear in top-k results
- Comparing sparse, dense, and hybrid candidate retrieval
- Detecting coverage regressions after reindexing or filter changes
- Diagnosing incomplete answers caused by missing evidence
- Setting retrieval gates for high-value or regulated questions
Example
Loading code...Frequently Asked Questions
Can Context Recall be measured without reference evidence?
Not as true recall, because the denominator is unknown. You can report observable proxies such as hit rate, retrieval consistency, or judge-estimated support, but label them clearly and avoid presenting them as coverage of all required evidence.
Is Context Recall the same as retriever recall at K?
They can be equivalent under an ID-based protocol where gold evidence IDs define relevance. A claim-based Context Recall metric instead asks which reference claims are supported by retrieved text, so its unit and judge error differ from document-level recall@K.
Does increasing top-k always improve Context Recall?
It can only recover evidence that the candidate retriever ranked within the expanded range, and it may add duplicates or irrelevant context. Measure context precision, latency, and downstream answer quality rather than treating a larger K as a free improvement.
How should unanswerable questions be handled?
Label whether the required evidence exists in the authorized corpus. If it does not, evaluate retrieval rejection and answer abstention instead of inventing required evidence or assigning a misleading recall score.
What should be investigated when Context Recall falls?
Compare source and index versions first, then inspect ingestion coverage, chunk boundaries, query normalization, metadata filters, candidate scores, rank depth, and authorization. Review misses by query class so a targeted repair is chosen instead of indiscriminately increasing top-k.