MLX-LM model architecture integration and loader contracts
Parent: MLX-LM native service model lifecycle and release contracts · Published reference · snapshot 2026-10-23 · viability 4.35 / 5 (research)
↓ Facts as markdownall context files
Source-anchored research for MLX-LM model architecture integration and loader contracts. Run status: COMPLETED.
These notes link each claim to its source. Some citations identify archived research reports that are no longer hosted on this site. A published reference means the content is available; it does not certify independent review or accuracy.Read the editorial policy and follow the sources before relying on a claim.
How it works
- A model-type alias is valid only when the model graph and weight semantics match. The oMLX Xing4.0 report describes a distinct Xing4_0HyperConnection residual topology and says aliasing xing4_0 to deepseek_v3 would produce incorrect outputs; it calls for a compatible implementation and sanitizer instead. (medium confidence) [source]
- MLX-LM imports a config-declared model_file from the model directory only when trust_remote_code=True; the default is false. The merged upstream change threads this opt-in through load() and sharded_load(), and oMLX preflights the same trust decision before starting MLX-LM loading. (high confidence) (verified as of 2026-10-23) [source]
Problems, failure modes and limitations
- load_model reads model*.safetensors, applies optional model_config overrides, builds ModelArgs.from_dict(config) and Model, calls model.sanitize(weights) when provided, then calls model.load_weights(..., strict=True) by default. A registry mapping alone therefore does not adapt checkpoint tensor names or layouts that the selected model cannot consume. (high confidence) (verified as of 2026-10-23) [source]
Comparisons and alternatives
- load_model accepts a get_model_classes callable that maps config to (Model, ModelArgs) and defaults to _get_classes. The higher-level load() signature does not expose or forward this callback, so it is a resolver hook for direct load_model callers rather than a global loader registry. (low confidence) (verified as of 2026-10-23) [source]
Facts and statements
- In MLX-LM's built-in path, _get_classes requires config['model_type'], optionally remaps a (model_type, architectures) pair or a model-type alias, imports mlx_lm.models.<resolved-type>, and expects that module to expose Model and ModelArgs. A repository name or architecture label alone does not add an implementation to the loader. (high confidence) (verified as of 2026-10-23) [source]
- load_model has a separate config model_file path: it imports that file and expects Model and ModelArgs. This route raises unless trust_remote_code=True, which defaults to false. A Qwen3.8 checkpoint repository demonstrates stock mlx_lm.load(..., trust_remote_code=True) using its bundled model_file for an architecture absent from the released registry. (high confidence) (verified as of 2026-10-23) [source]
- Model-port validation needs reference outputs as well as model-type and shape checks: porting guidance describes tensor transposes and rotary/cache conventions that preserve shapes but change outputs, while an independent Qwen3.8 port reports sanitization, normalization, hash-seed, and sparse-mask errors in a candidate upstream implementation. (high confidence) (verified as of 2026-10-23) [source]
- The minimum architecture metadata is consumer-specific: MLX-LM requires model_type and treats architectures as optional remapping input, while Ollama issue #19128 reports that its MLX importer and runner rejected a checkpoint with model_type but no architectures[0], even though the model_type named a supported runner architecture. Services should define and validate metadata fallbacks for each backend. (medium confidence) (verified as of 2026-10-19) [source]
- _get_classes reads config.model_type and examines architectures in order, treating a string as one architecture. The first matching (model_type, architecture) entry in MODEL_ARCHITECTURE_REMAPPING wins; if none matches, MODEL_REMAPPING applies. It imports mlx_lm.models.<resolved_type> and returns that module's Model and ModelArgs. ImportError becomes ValueError("Model type <resolved_type> not supported."). (low confidence) (verified as of 2026-10-23) [source]
- load_model reads config.json, then calls config.update(model_config) for supplied overrides before ModelArgs.from_dict and Model construction. This is a shallow dictionary update: supplied values replace values under matching top-level keys. A model sanitize hook transforms the loaded weight dictionary before the final model.load_weights call. (low confidence) (verified as of 2026-10-23) [source]
- With strict=True, load_model raises FileNotFoundError("No safetensors found in <model_path>") if model*.safetensors matches no files. It passes strict to model.load_weights; the documented contract raises if weights do not match the model. strict=False relaxes that contract and the missing-weight-files guard. (low confidence) (verified as of 2026-10-23) [source]
- For config.quantization, the loader first honors explicit per-module entries keyed by module path. Otherwise class_predicate requires a to_quantized method and a matching <path>.scales checkpoint tensor, then infers the module quantization settings from checkpoint shapes/dtypes. A supported global_scale tensor adds that argument. This selective conversion prepares modules before weights are loaded. (low confidence) (verified as of 2026-10-23) [source]
Children
- MLX-LM built-in architecture dispatch (frontier)
- MLX-LM model_file and trust_remote_code contract (frontier)
- MLX-LM architecture-specific weight sanitization and parity validation (frontier)
Frontier under this node: MLX-LM architecture-specific weight sanitization and parity validation, MLX-LM built-in architecture dispatch, MLX-LM model_file and trust_remote_code contract