What is Hallucination?
Hallucination is confidently stated generated content that is erroneous or false relative to a declared truth or evidence boundary. Production evaluation should classify atomic claims as supported, contradicted, insufficient evidence, or not applicable rather than judging an entire answer from fluency.
Quick Facts
| Full Name | AI Hallucination |
|---|---|
| Created | Term popularized in AI context around 2020-2022 with LLM adoption |
How It Works
AI hallucination, called confabulation in NIST AI 600-1, is a factuality risk rather than a writing-style signal. A useful test first defines the task's truth mode, source authority, evidence revision, validity time, and access scope. It then extracts atomic claims, links each claim to canonical evidence, assigns a bounded verdict, and applies a task-specific decision policy. Contradicted and insufficient-evidence claims must remain separate: missing proof does not establish falsehood. Retrieval-Augmented Generation can supply current and citable evidence, but retrieval misses, stale sources, conflicts, generation drift, and invalid citations still require independent measurement. Lower temperature, structured output, model agreement, and self-reported confidence can improve reproducibility or triage, but none proves truth. High-impact systems need calibrated judges, appropriate abstention, qualified review, and deterministic authorization for external actions.
Key Characteristics
- Boundary-relative: a factual verdict names the truth source, revision, time, jurisdiction, or evidence contract
- Claim-level: mixed answers are decomposed into atomic factual claims before evaluation
- Four-state: supported, contradicted, insufficient evidence, and not applicable remain distinct
- Evidence-traceable: verdicts preserve canonical source IDs, revisions, authority, and citation anchors
- Risk-sensitive: release, revise, abstain, or review decisions depend on claim materiality and task loss
- System-level: corpus, retrieval, ranking, generation, citation, and decision stages can each introduce failure
Common Use Cases
- Detecting fabricated citations, entities, dates, quantities, or quotations in generated answers
- Evaluating RAG answer faithfulness separately from retrieval coverage and external correctness
- Routing material claims with insufficient evidence to abstention or qualified human review
- Gating model, prompt, retriever, corpus, or judge changes before production release
- Investigating factuality incidents with versioned evidence and decision traces
Example
Loading code...Frequently Asked Questions
What is the difference between a contradicted and unsupported claim?
A contradicted claim conflicts with evidence that supports an incompatible conclusion. An unsupported or insufficient-evidence claim cannot be established from the declared evidence snapshot. It may be true outside that snapshot, so systems should retrieve, abstain, or review instead of automatically calling it false.
How do you measure LLM hallucination?
Extract and label atomic claims, then report contradiction and insufficient-evidence precision and recall, citation precision and coverage, correctness, faithfulness, abstention precision and recall, selective risk, and answer coverage. Always publish denominators and results by risk slice.
Does RAG eliminate hallucinations?
No. RAG provides evidence, provenance, and fresher knowledge, but the corpus may be wrong or stale, retrieval can miss required evidence, and generation can distort or ignore context. Evaluate corpus, retrieval, ranking, faithfulness, citations, and release decisions separately.
Does a lower temperature reduce hallucination?
A lower temperature reduces sampling variation and can make a run easier to reproduce. It does not verify a claim, and a deterministic model can repeatedly produce the same false statement. Use evidence-based evaluation independently of sampling settings.
Can an LLM judge detect hallucinations automatically?
An LLM judge can scale claim extraction and evidence comparison, but it can omit claims, misread evidence, and show position or style bias. Require typed verdicts and evidence IDs, calibrate against qualified human labels, monitor each task slice, and keep deterministic release rules.