Pre-M5 vs M5 Apple GPU decode behavior for compressed KV
Parent: Mac local LLMs: KV cache sizing and quantization · Published reference · snapshot 2026-10-05
↓ Facts as markdownall context files
Apple's MLX study reports M5 generation is 19-27% faster than M4, explained by memory bandwidth (120 GB/s M4, 153 GB/s M5, +28%), while TTFT is compute-bound and up to 4x faster with the GPU Neural Accelerators.
These notes link each claim to its source. A source may be a research report hosted on this site rather than the primary document. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
Facts
- Apple's MLX study reports M5 generation is 19-27% faster than M4, explained by memory bandwidth (120 GB/s M4, 153 GB/s M5, +28%), while TTFT is compute-bound and up to 4x faster with the GPU Neural Accelerators. [source]
- Inference: because M5 accelerators speed matmul but not decode bandwidth, a compressed-KV dequant penalty on pre-M5 ALU-bound GPUs (M1 L2 wall) is not expected to be rescued by the accelerators; no measurement of turbo/q4 KV on M5 vs M1 with identical models was found. [source]
Children
- No children recorded.