PagedAttention (vLLM): Block Table + Copy-on-Write + KV-Cache Fragmentation

1. PagedAttention Konsepti#

• Her request'in KV-cache'i contiguous tensor
• Request'in beklenen max-length boyunca pre-allocate
• Fragmentation: %30-40 boşa giden bellek

• KV-cache 16-token block'lara bölünmüş
• Her request'in block table'i var (logical idx → physical block ptr)
• Block-level allocation → no fragmentation
• Copy-on-write: prefix cache (system prompt) ile multiple request aynı block'a pointer

Request A: "You are a helpful assistant. Hello, my name is Ahmet."
Request B: "You are a helpful assistant. What is 2+2?"

Block 0: "You are a helpful" → shared between A & B (ref count 2)
Block 1: "assistant." → shared
Block 2 (A): "Hello, my name is Ahmet."
Block 2 (B): "What is 2+2?"