What is RAG?
RAG (Retrieval-Augmented Generation) is an architecture that retrieves external evidence at inference time and supplies selected context to a generative model so its answer can be grounded, updated, and attributed.
Quick Facts
| Full Name | Retrieval-Augmented Generation |
|---|---|
| Created | 2020 by Facebook AI Research (Lewis et al.) |
| Specification | Official Specification |
How It Works
From source record to answer
The offline path parses sources, preserves stable document IDs and versions, applies access metadata, and creates chunks or other retrieval units. The online path interprets the query, enforces filters, retrieves candidates, optionally reranks or compresses them, assembles a bounded context, generates an answer, and maps answer claims back to evidence. Dense retrieval is only one option; sparse, hybrid, graph, SQL, and tool-backed retrieval can all serve the same evidence contract.
Diagnose the stage, not just the final answer
A wrong answer does not identify its own root cause. First test whether the corpus contains current authorized evidence, then whether retrieval covered that evidence, whether ranking kept useful passages near the top, whether the generator stayed faithful to supplied context, and whether citations support the claims they annotate. End-to-end task success matters, but stage metrics make regressions actionable.
Production controls and trade-offs
Chunking, top-k, filters, query rewriting, and reranking must be tuned on representative questions rather than fixed folklore values. Log source and index versions, retrieved IDs and scores, applied filters, model and prompt versions, citations, latency, and fallback decisions without leaking protected content. Use RAG when facts must change independently of model weights; use fine-tuning for behavior or representation changes, and combine them only when both needs are proven.
Key Characteristics
- Separates an updateable evidence store from the model's parametric knowledge
- Uses retrieval and ranking before generation rather than treating search as an afterthought
- Requires stable provenance, source versions, and access controls for auditable answers
- Exposes distinct indexing, retrieval, generation, and attribution failure modes
- Supports dense, sparse, hybrid, graph, database, and tool-backed evidence
- Needs stage-level and end-to-end evaluation because retrieval alone does not guarantee truth
Common Use Cases
- Answering enterprise questions from versioned internal policies and runbooks
- Supporting customers with current product, entitlement, and incident information
- Exploring legal, compliance, or research collections with traceable evidence
- Building coding assistants over repositories and technical documentation
- Generating evidence-backed summaries from frequently changing records
Example
Loading code...Frequently Asked Questions
What problem does RAG solve?
RAG lets an application consult changeable or private evidence without encoding every fact in model weights. It is most useful when users need current domain knowledge, inspectable sources, or access-controlled answers; it is not a substitute for teaching the model a new behavior.
Does RAG guarantee factual or hallucination-free answers?
No. The retriever may miss the required evidence, return misleading sources, or provide noisy context, and the generator may still add unsupported claims. Factuality also depends on source truth and freshness, so teams must evaluate retrieval, faithfulness, citation support, and task correctness separately.
How should chunk size and top-k be chosen?
Treat both as evaluated parameters, not universal constants. Build representative questions with required evidence, compare multiple chunking and ranking configurations, inspect context precision and recall, and include generation quality, latency, and token cost before selecting a configuration.
When should I use RAG instead of fine-tuning?
Choose RAG when the main requirement is retrieving facts that are private, frequently updated, or must be cited. Choose fine-tuning when the model must consistently follow a new style, format, or task behavior. Some systems need both, but each component should solve a demonstrated problem.
What should a production RAG trace record?
Record the query transformation, corpus and index versions, access filters, retrieved and reranked IDs with scores, context sent to the model, prompt and model versions, answer claims, citation mappings, fallback path, latency, and errors. Apply the same authorization and retention rules to traces as to source content.