LLM Compression (Quantization, Distillation, Pruning, Merging)
Parent: LLM Models and APIs · researched 2026-05-31T20:39:48.899Z· 30 sources · 13 concepts · skill ai-agent-engineering
AI & agent-engineering family ROUTER. Split into: ai-agents-orchestration (agent frameworks, multi-agent, memory, planning, guardrails, coding/GUI agents, autonomous loops, eval); ai-rag-retrieval (RA
ai-agent-engineering
- AI & agent-engineering family ROUTER. Split into: ai-agents-orchestration (agent frameworks, multi-agent, memory, planning, guardrails, coding/GUI agents, autonomous loops, eval); ai-rag-retrieval (RAG, iterative retrieval, vector/graph datastores); ai-llm-model-layer (training, fine-tuning, alignment/RLHF, compression, inference serving, transformer/multimodal architecture, model selection, observability); ai-mcp-sdk-prompting (MCP servers/builder, Anthropic SDK, prompt engineering, context engineering, LLM frameworks, tool-search, prompt lookup). Route to the matching sub-hub. [source]
- This hub routes to on-demand reference files under references/. See each spoke for depth. [source]
Children
- PTQ vs QAT (frontier)
- GPTQ (frontier)
- AWQ (frontier)
- SmoothQuant (W8A8 activation quant) (frontier)
- GGUF llama.cpp k-quants & imatrix (frontier)
- bitsandbytes NF4 & LLM.int8 (frontier)
- FP8/INT4 & MX/MXFP4 microscaling (frontier)
- KV-cache quantization (KIVI/KVQuant) (frontier)
- BitNet & EfficientQAT (frontier)
- Knowledge distillation (logit/feature/sequence/on-policy GKD/MiniLLM/DistiLLM) (frontier)
- Pruning & sparsity (SparseGPT, Wanda, 2:4 N:M) (frontier)
- Model merging (TIES/DARE/SLERP/task-arithmetic/soups/MergeKit) (frontier)
- Compressed-model evaluation (perplexity vs KL vs flips) (frontier)
Frontier under this node: AWQ, BitNet & EfficientQAT, Compressed-model evaluation (perplexity vs KL vs flips), FP8/INT4 & MX/MXFP4 microscaling, GGUF llama.cpp k-quants & imatrix, GPTQ, KV-cache quantization (KIVI/KVQuant), Knowledge distillation (logit/feature/sequence/on-policy GKD/MiniLLM/DistiLLM), Model merging (TIES/DARE/SLERP/task-arithmetic/soups/MergeKit), PTQ vs QAT, Pruning & sparsity (SparseGPT, Wanda, 2:4 N:M), SmoothQuant (W8A8 activation quant), bitsandbytes NF4 & LLM.int8