<!-- llms-explorer concept facts · https://llms-explorer.com/tree/mlx-kv-cache-quantization-and-zero-copy-pointer-passing/ · pack 2026-09-30 · ~252 tokens -->

# MLX KV Cache Quantization and Zero-Copy Pointer Passing

> Source-anchored research for MLX KV Cache Quantization and Zero-Copy Pointer Passing. Run status: COMPLETED.

Parent: [Local Inference Acceleration: Memory Architectures, Metal Kernels, and Latency Optimization](https://llms-explorer.com/tree/local-inference-acceleration-and-kernels/) · 2 facets · 2 facts · page: https://llms-explorer.com/tree/mlx-kv-cache-quantization-and-zero-copy-pointer-passing/

## Measurements and reference values

- Quantizing KV cache tensors to 4-bit or 8-bit in MLX reduces context memory growth from ~9.0 GB to ~2.5 GB at 64k tokens on 35B parameter models with negligible loss in retrieval recall. (high confidence) — [source](https://github.com/ml-explore/mlx)

## Facts and statements

- MLX allocates tensor buffers directly into unified memory pages mapped with shared CPU-GPU storage modes, eliminating DMA transfers and serialization overhead. (high confidence) — [source](https://github.com/ml-explore/mlx)
