<!-- llms-explorer concept facts · https://llms-explorer.com/tree/ttft-compute-bound-analysis-and-inter-token-latency-jitter/ · pack 2026-09-30 · ~245 tokens -->

# TTFT Compute Bound Analysis and Inter-token Latency Jitter

> Source-anchored research for TTFT Compute Bound Analysis and Inter-token Latency Jitter. Run status: COMPLETED.

Parent: [Local Inference Acceleration: Memory Architectures, Metal Kernels, and Latency Optimization](https://llms-explorer.com/tree/local-inference-acceleration-and-kernels/) · 2 facets · 2 facts · page: https://llms-explorer.com/tree/ttft-compute-bound-analysis-and-inter-token-latency-jitter/

## How it works

- Inter-token latency variance (jitter) is primarily induced by dynamic KV cache heap allocations and memory compaction pauses when context thresholds are crossed. (high confidence) — [source](https://github.com/vllm-project/vllm)

## Facts and statements

- TTFT scales with prompt length in proportion to compute matrix FLOPs (O(N)), while sequential token generation scales purely with memory bandwidth per token (O(1)). (high confidence) — [source](https://github.com/vllm-project/vllm)
