What is PagedAttention?
Updated on
PagedAttention is an LLM serving memory-management and attention approach that maps fixed-size logical KV Cache blocks for each sequence to non-contiguous physical blocks allocated on demand.
Quick Facts
| Created | 2023 by the vLLM authors at UC Berkeley and collaborators |
|---|---|
| Specification | Official Specification |
How It Works
Map Logical to Physical Blocks
The scheduler assigns each sequence logical block indices 0..n. A Block Table maps each logical index to a physical block ID, so blocks for one request need not be adjacent in GPU memory. Prefill allocates enough blocks for the input; Decode appends into the partially filled tail and requests another block only when it is full. Allocation failure must trigger admission control, preemption, swap or rejection explicitly rather than corrupting another sequence's state.
Share Without Mutating State
Multiple sequences can reference immutable physical blocks for a shared Prefix or parallel-sampling branch. Reference counts govern reclamation. If one sequence must write into a shared partial block, Copy-on-write allocates a private block, copies valid entries and updates only that sequence's Block Table. Cache identity must include the exact model, Tokenizer, Template, Adapter, input Tokens and relevant execution settings; matching text alone is not a safe sharing key.
Separate the Idea from One Kernel
Paged KV management, the attention kernel and the serving scheduler are related but distinct. The current [vLLM Paged Attention page](https://docs.vllm.ai/en/stable/design/paged_attention/) explicitly labels its walkthrough historical and says it no longer describes today's code. Other systems may use paged caches with different layouts, while [vAttention](https://arxiv.org/abs/2405.04437) keeps contiguous virtual addresses and maps physical pages on demand. Benchmark allocation waste, page faults, preemption, TTFT, TPOT and Goodput on the pinned implementation.
Key Characteristics
- Splits each sequence's KV Cache into fixed-size logical Token blocks
- Maps logical order to non-contiguous physical GPU blocks through a Block Table
- Allocates and reclaims capacity as sequences grow, finish or are preempted
- Can share immutable Prefix blocks and isolate writes with Copy-on-write
- Reduces reservation and fragmentation waste but retains tail-block and metadata overhead
- Improves capacity only when the current allocator, kernel and scheduler make paging worthwhile
Common Use Cases
- Serving variable-length requests when KV Cache limits active Concurrency
- Supporting Continuous Batching without maximum-length allocation per request
- Sharing immutable Prefix state across parallel samples or repeated prefixes
- Tracking block pressure for admission, preemption and cache eviction decisions
- Comparing paged and virtual-memory KV layouts on a pinned serving Runtime
Example
Loading code...Frequently Asked Questions
How does PagedAttention reduce KV Cache waste?
It allocates fixed-size physical blocks as a sequence grows instead of reserving one maximum-length contiguous buffer. A Block Table preserves logical order, so free blocks can be used even when their physical addresses are not adjacent.
Is PagedAttention the same as FlashAttention?
No. PagedAttention primarily addresses KV Cache placement and access during serving. FlashAttention is an IO-aware attention algorithm and kernel family. A serving Runtime may combine paged KV storage with a compatible optimized attention kernel.
Does PagedAttention eliminate all memory fragmentation?
No. It greatly reduces large contiguous reservations and external fragmentation, but the last block of each sequence can be partially empty. Block metadata, allocator reserves and Runtime buffers also consume memory.
Why does PagedAttention use Copy-on-write?
Shared Prefix or branch blocks must remain immutable for other sequences. When one sequence writes to a shared partial block, Copy-on-write gives it a private copy while preserving the original references.
Does the original PagedAttention paper describe current vLLM internals exactly?
No. The paper defines the foundational design, but vLLM's own stable documentation labels its kernel walkthrough historical. Treat current block layout, kernels and performance as versioned implementation details and verify the deployed release.