MLX-LM adapter checkpoints fusion and export
Parent: MLX-LM training and adapter lifecycle · Published reference · snapshot 2026-10-23 · viability 4.65 / 5 (research)
↓ Facts as markdownall context files
Source-anchored research for MLX-LM adapter checkpoints fusion and export. Run status: COMPLETED.
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
How it works
- The current `mlx_lm.fuse` command loads a base model and adapter directory, fuses model modules that expose a `fuse` method, and saves the combined model at `--save-path`; the defaults are `mlx_model`, `adapters`, and `fused_model`. It can also upload the saved model when given `--upload-repo`. Apple describes the fused result as a self-contained model for simpler deployment and sharing. (high confidence) (verified as of 2026-10-23) [source]
How-to and procedures
- Use an unfused adapter for generation with mlx_lm.generate --model <base_model> --adapter-path <adapter_directory> --prompt "<prompt>". LORA.md documents --upload-repo as the fused-model upload destination and --hf-path as the original model for its card, but the pinned fuse.py parser has no --hf-path option. Treat that documented upload invocation as a guide/parser disagreement and inspect the installed mlx_lm.fuse --help before publishing. (high confidence) (verified as of 2026-10-23) [source]
Problems, failure modes and limitations
- On `mlx-lm` 0.31.3 with `mlx` 0.32.0, issue #1723 reported `[save_gguf] can only serialize row-major arrays` during `--export-gguf` because `save()` ignored `donate_model=False` and emptied weights before conversion. Upstream merged PR #1743 on 2026-08-17 to pass that flag through. Treat the error as version-specific and check the installed build before reapplying workarounds. (high confidence) (verified as of 2026-08-17) [source]
- A 2026 llama.cpp issue reports that its separate `convert_hf_to_gguf.py` path rejected `lm_head.scales` and sometimes `lm_head.biases` in an MLX-fused artifact. That report is specific to the downstream converter and does not establish a failure in MLX-LM's built-in `--export-gguf` implementation. (low confidence) [source]
Comparisons and alternatives
- --resume-adapter-file loads model weights with model.load_weights(..., strict=False). It does not restore optimizer state or a previous iteration counter in this entry point, so it is a weight-based continuation rather than a complete training-state restart. (low confidence) (verified as of 2026-10-23) [source]
Facts and statements
- Current MLX-LM training writes the current adapter weights to `adapters.safetensors` and, at each configured save interval, also writes a numbered `<step>_adapters.safetensors` snapshot beside it. The loader called by fusion reads `adapter_config.json` and `adapters.safetensors` from the `--adapter-path` directory, so using a numbered snapshot requires exposing the chosen weights under the expected `adapters.safetensors` name beside the corresponding config. (high confidence) (verified as of 2026-10-23) [source]
- The current `--export-gguf` path accepts only `model_type` values `llama`, `mixtral`, and `mistral`; the docs also scope support to those model styles at fp16 and give `fused_model/ggml-model-f16.gguf` as the default output, overridable with `--gguf-path`. The optional `--dequantize` path dequantizes before saving and removes quantization metadata from the saved config. A `gpt_oss` export rejection was reported through the same model-type gate. (high confidence) (verified as of 2026-10-23) [source]
- Two case reports show quantized fusion can change observed behavior: an independent test found a rank-8 adapter's behavior disappeared after fusion into a 4-bit base without `--dequantize`, while issue #654 reports that direct fusion changed hidden reasoning for `gpt_oss-20b` and that dequantizing before a separate 4-bit conversion restored it. These reports are model-specific and do not establish a general degradation rate; evaluate the target model after fusion. (medium confidence) (verified as of 2026-07-30) [source]
- The --adapter-path option chooses the adapter directory. Training creates that directory and writes adapters.safetensors plus adapter_config.json there; --save-every controls the periodic checkpoint interval by populating TrainingArgs.steps_per_save. (high confidence) (verified as of 2026-10-23) [source]
Children
- No children recorded.