What is Lost in the Middle?
Lost in the Middle is the tendency of language models to use information near the beginning or end of a long context more reliably than information placed in the middle.
How It Works
Lost in the middle is a practical long-context failure mode. Even when a model's context window is large enough to contain all evidence, it may not attend to middle-position information as reliably as information near the start or end. This matters for RAG, agent memory, legal review, and long document QA. Mitigation usually involves better ranking, context compression, section ordering, summaries, citations, and evaluation cases that test where evidence appears, not only whether it appears.
Key Characteristics
- Appears when important evidence is buried inside long prompts
- Can occur even when the model technically supports the context length
- Affects RAG answers, long document QA, agent memory, and multi-document prompts
- Depends on model architecture, training, prompt structure, and evidence position
- Requires evaluation that varies evidence placement within the context
Common Use Cases
- Testing whether RAG answers use evidence placed in the middle of context
- Reordering retrieved chunks to put critical evidence near the model's focus areas
- Compressing long contexts before generation
- Designing long-document QA prompts with section summaries
- Diagnosing missed citations despite evidence being retrieved
Example
Loading code...Frequently Asked Questions
Does a larger context window solve lost in the middle?
Not by itself. Larger windows allow more text, but models may still underuse evidence depending on placement and prompt structure.
How can lost in the middle be detected?
Use evaluation cases where the same evidence appears at the beginning, middle, and end of the context and compare answer quality.
How can teams reduce this problem?
Rank evidence carefully, compress context, repeat key facts in summaries, use citations, and avoid stuffing irrelevant chunks.
Why does this matter for RAG?
A retriever may return the right evidence, but the generator can still miss it if the evidence is buried in a long context.