Asymmetric KV Cache Quantization Algorithms
Parent: Local Inference Micro-Architectures: Signaling, Quantization Superblocks, and Kernel Pipeline Scheduling · Published reference · snapshot 2026-09-30
↓ Facts as markdownall context files
Source-anchored research for Asymmetric KV Cache Quantization Algorithms. Run status: COMPLETED.
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
Structure and components
- Key vectors in transformer self-attention contain sensitive channel-specific magnitude spikes that cause severe perplexity degradation if quantized uniformly with standard symmetric rounding. (high confidence) [source]
Measurements and reference values
- Asymmetric KV cache quantization preserves key vector outlier dimensions in FP8 or FP16 while quantizing value vectors to 4-bit affine blocks, reducing KV memory by up to 70% with less than 0.1 perplexity delta. (high confidence) [source]
Children
- No children recorded.