# GQA KV Cache Memory Mechanics > Source-anchored research for GQA KV Cache Memory Mechanics. Run status: COMPLETED. Parent: [16 GB VRAM residency budgeting](https://llms-explorer.com/tree/16-gb-vram-residency-budgeting/) · 3 facets · 3 facts · page: https://llms-explorer.com/tree/gqa-kv-cache-memory-mechanics/ ## Structure and components - For Qwen2-14B with 48 layers, 8 KV heads, and 128 head dimension at FP16, KV cache consumes exactly 196.6 KB per token, or 3.14 GB for a 16,384-token context window. (high confidence) — [source](https://arxiv.org/abs/2305.13245) ## Measurements and reference values - FP8 KV cache quantization reduces per-token KV memory by 50% down to 98.3 KB per token, permitting up to 32,768 context tokens in 3.14 GB on 16 GB accelerators. (medium confidence) — [source](https://github.com/vllm-project/vllm) ## Facts and statements - In GQA architectures such as Qwen2-14B and Llama-3.1-8B, KV cache memory footprint scales as 2 * num_layers * num_kv_heads * head_dim * precision_bytes per context token. (high confidence) — [source](https://arxiv.org/abs/2305.13245)