# Gemma 4 shared KV cache layers YOCO > Gemma 4 E2B has 35 transformer layers of which only the first 15 compute their own K/V; the final 20 reuse K/V from the most recent earlier non-shared layer of the same attention type. Parent: [Mac local LLMs: KV cache sizing and quantization](https://llms-explorer.com/tree/mac-local-llms-kv-cache-sizing-and-quantization/) · 2 facets · 8 facts · page: https://llms-explorer.com/tree/gemma-4-shared-kv-cache-layers-yoco/ ## Facts - Gemma 4 E2B has 35 transformer layers of which only the first 15 compute their own K/V; the final 20 reuse K/V from the most recent earlier non-shared layer of the same attention type. — [source](https://sebastianraschka.com/llm-architecture-gallery/kv-sharing/) - Gemma 4 E4B has 42 layers, 24 computing their own K/V and the final 18 sharing. — [source](https://sebastianraschka.com/llm-architecture-gallery/kv-sharing/) - Raschka's estimate: sharing saves roughly half the remaining cache after GQA/MQA, about 2.7 GB (E2B) and 6 GB (E4B) at bfloat16 and 128k context. — [source](https://sebastianraschka.com/llm-architecture-gallery/kv-sharing/) - Raschka's plotted E2B-like setup falls from 37.58 GB (MHA) to 2.01 GB (MQA plus sharing, 15 producers) at 128k, and E4B-like from 56.37 GB to 8.05 GB (GQA plus sharing, 24 producers), before sliding-window retention savings. — [source](https://sebastianraschka.com/llm-architecture-gallery/kv-sharing/) - Cross-layer KV sharing was not invented by Gemma 4; Brandon et al., Reducing Transformer Key-Value Cache Size with Cross-Layer Attention (NeurIPS 2024), introduced it, and Gemma 4 E2B/E4B is the first popular architecture cited using it. — [source](https://sebastianraschka.com/llm-architecture-gallery/kv-sharing/) - Shared layers still compute their own queries, so each layer can form its own attention pattern while reusing producer K/V. — [source](https://sebastianraschka.com/llm-architecture-gallery/kv-sharing/) - botmonster states the sharing applies to the 26B and 31B models and that 10 of 30 shared layers would cut KV memory about 33%. — [source](https://botmonster.com/ai/gemma-4-architecture-per-layer-embeddings-shared-kv-cache-dual-rope/) ## Corrections and disagreements - CONTRADICTS botmonster claim above: llama.cpp metadata for 26B-A4B reports shared_kv_layers = 0, so for that model the 26B-A4B cache arithmetic must use all attention layers, not a reduced producer count (see hybrid-and-sliding-window-attention-kv-cache-rewinding.md). — source: `asserted`