Mac local LLMs: llama.cpp internals

Parent: Running LLM models locally on a Mac · Published reference · snapshot 2026-10-05

↓ Facts as markdownall context files

Metal is ON by default on Apple (GGML_METAL_DEFAULT ON); plain `cmake -B build && cmake --build build --config Release`; `-DGGML_METAL=OFF` or `-ngl 0` disables.

These notes link each claim to its source. A source may be a research report hosted on this site rather than the primary document. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.

Children

• Metal residency sets and llama.cpp Metal internals
• mmproj versioning and mismatch errors for multimodal GGUFs
• mtmd mergeable bitmaps and temporal frame merging for Qwen-VL video
• mtmd_tokenize_from_parts per-segment tokenization for marked input
• Jinja templates failing on llama.cpp engine but passing in Python
• llama.cpp causal_attn flag scheduler re-reserve cost per image (PR 28751)
• llama.cpp chat.cpp per-template regex rewrites versus Jinja input marking
• llama.cpp context checkpoint 64-token minimum tuning
• llama.cpp context checkpoints and SWA hybrid prompt re-processing
• llama.cpp --fit auto memory fitting
• llama.cpp gemma4uv projector type support and first supporting build
• llama.cpp jinja capability probes chat_template_caps
• llama.cpp Jinja input marking is_input special-token injection defence
• llama.cpp Jinja runtime (minja replacement) tojson and filter semantics
• llama.cpp message_spans chat-template user-boundary detection for checkpoints
• llama.cpp Metal backend on Mac
• llama.cpp --mtmd-batch-max-tokens tuning against n_ubatch for non-causal project
• llama.cpp mtmd batching of consecutive image chunks into one non-causal decode
• llama.cpp mtmd lazy bitmaps for frame-by-frame video
• llama.cpp mtmd video and webp input via ffmpeg subprocess (MTMD_VIDEO, PR 27520)
• llama.cpp PR 25592 exact-position hybrid checkpoint restore
• llama.cpp reasoning-preserve flag and template support detection
• llama.cpp seq_pos_min hybrid memory fix PR 24797
• llama.cpp slot state trimming when the template will drop reasoning
• llama-fit-params tool and freezing fitted flags into a launch script
• llama_n_ctx_seq semantics with and without unified KV
• llama-server /props n_ctx as the per-slot window to advertise to clients
• llama-server router child bind address versus router --host
• llama-server router subprocess build option (subprocess is not enabled on this b
• llama-server slot and unified-KV configuration
• llama-server --video-fps, --video-timestamp-interval and --video-ffmpeg-dir flag

← the whole tree · 3D view· how to read this page