GEMM Compute Scaling vs GEMV Memory-Bandwidth Bounds
Parent: Local Inference Micro-Architectures: Signaling, Quantization Superblocks, and Kernel Pipeline Scheduling · Published reference · snapshot 2026-09-30
↓ Facts as markdownall context files
Source-anchored research for GEMM Compute Scaling vs GEMV Memory-Bandwidth Bounds. Run status: COMPLETED.
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
Facts and statements
- Prompt pre-fill evaluates GEMM operations where high arithmetic intensity (FLOPs/byte > 100) saturates GPU tensor cores and scales linearly with compute clock frequencies. (high confidence) [source]
- Auto-regressive decoding executes GEMV operations with minimal arithmetic intensity (FLOPs/byte approx. 1), causing GPU compute ALUs to sit idle while awaiting memory bus DRAM/GDDR reads. (high confidence) [source]
Children
- No children recorded.