Gemma 4 shared KV cache layers YOCO
Parent: Mac local LLMs: KV cache sizing and quantization · Published reference · snapshot 2026-10-05
↓ Facts as markdownall context files
Gemma 4 E2B has 35 transformer layers of which only the first 15 compute their own K/V; the final 20 reuse K/V from the most recent earlier non-shared layer of the same attention type.
These notes link each claim to its source. A source may be a research report hosted on this site rather than the primary document. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
Facts
- Gemma 4 E2B has 35 transformer layers of which only the first 15 compute their own K/V; the final 20 reuse K/V from the most recent earlier non-shared layer of the same attention type. [source]
- Gemma 4 E4B has 42 layers, 24 computing their own K/V and the final 18 sharing. [source]
- Raschka's estimate: sharing saves roughly half the remaining cache after GQA/MQA, about 2.7 GB (E2B) and 6 GB (E4B) at bfloat16 and 128k context. [source]
- Raschka's plotted E2B-like setup falls from 37.58 GB (MHA) to 2.01 GB (MQA plus sharing, 15 producers) at 128k, and E4B-like from 56.37 GB to 8.05 GB (GQA plus sharing, 24 producers), before sliding-window retention savings. [source]
- Cross-layer KV sharing was not invented by Gemma 4; Brandon et al., Reducing Transformer Key-Value Cache Size with Cross-Layer Attention (NeurIPS 2024), introduced it, and Gemma 4 E2B/E4B is the first popular architecture cited using it. [source]
- Shared layers still compute their own queries, so each layer can form its own attention pattern while reusing producer K/V. [source]
- botmonster states the sharing applies to the 26B and 31B models and that 10 of 30 shared layers would cut KV memory about 33%. [source]
Corrections and disagreements
- CONTRADICTS botmonster claim above: llama.cpp metadata for 26B-A4B reports shared_kv_layers = 0, so for that model the 26B-A4B cache arithmetic must use all attention layers, not a reduced producer count (see hybrid-and-sliding-window-attention-kv-cache-rewinding.md). [source]
Children
- No children recorded.