llama.cpp --mtmd-batch-max-tokens tuning against n_ubatch for non-causal project
Parent: Mac local LLMs: llama.cpp internals · Published reference · snapshot 2026-10-05
↓ Facts as markdownall context files
Already covered: llama-cpp-mtmd-batching-of-consecutive-image-chu.md holds the default 1024, the not-a-hard-limit first-image rule, the `--mtmd-batch-max-tokens` flag, and the rule that for non-causal models `max_tokens` must not exceed `n_ubatch`. The header comment on `mtmd_memory_usage` and th...
These notes link each claim to its source. A source may be a research report hosted on this site rather than the primary document. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
Facts
- Already covered: llama-cpp-mtmd-batching-of-consecutive-image-chu.md holds the default 1024, the not-a-hard-limit first-image rule, the `--mtmd-batch-max-tokens` flag, and the rule that for non-causal models `max_tokens` must not exceed `n_ubatch`. The header comment on `mtmd_memory_usage` and the `batch_max_tokens` comment add nothing beyond that; no source gives a tuning guideline relating the two. [source]
Children
- No children recorded.