# MLX-LM training configuration optimization and metrics > Source-anchored research for MLX-LM training configuration optimization and metrics. Run status: COMPLETED. Parent: [MLX-LM training and adapter lifecycle](https://llms-explorer.com/tree/mlx-lm-training-and-adapter-lifecycle/) · 4 facets · 15 facts · page: https://llms-explorer.com/tree/mlx-lm-training-configuration-optimization-and-metrics/ ## Structure and components - The current MLX Adam and AdamW implementations default bias_correction to false. Treat optimizer settings as part of the run configuration when comparing learning-rate experiments or porting settings from another framework. (high confidence) (verified as of 2026-10-23) — [source](https://github.com/ml-explore/mlx/blob/main/python/mlx/optimizers/optimizers.py) ## How it works - The current schedule builder uses the named MLX optimizer schedule with the configured arguments; when warmup is enabled, it linearly ramps from warmup_init to arguments[0] and joins the main schedule at warmup_steps + 1. Inspect the logged learning rate to verify that the configured schedule follows the intended trajectory. (high confidence) (verified as of 2026-10-23) — [source](https://github.com/ml-explore/mlx-lm/blob/main/mlx_lm/tuner/utils.py) - Load YAML training settings with --config or -c . main() copies each YAML value only when the parsed CLI value for that key is None, then fills remaining None values from CONFIG_DEFAULTS. Explicit non-None CLI values take precedence; parser defaults that are already non-None can also block a YAML override. (low confidence) (verified as of 2026-10-23) — [source](https://raw.githubusercontent.com/ml-explore/mlx-lm/0d4d3d04f8ab7ce4a51e7209e8615755a48ba94d/mlx_lm/lora.py) ## Comparisons and alternatives - Learning-rate selection is consequential and should be tested for the model and run: an LLM learning-rate study argues that policies designed for traditional DNN training may not transfer directly, and a reported mlx-lm 0.31.3 case saw one 1e-4 run produce 162 errors on a held-out task versus 18 errors in a later 1e-5 run. The author notes that the example order also changed and there was only one run per setting, so it does not establish a universal rate or isolate learning rate as the sole cause. (medium confidence) — [source](https://arxiv.org/abs/2309.08859) - An mlx-lm issue reports that the same stated data and configuration produced different training and validation loss behavior after upgrading from 0.21.5 to 0.22.0. The thread does not establish the cause, so this is a version-specific user report rather than evidence of a confirmed regression. (low confidence) — [source](https://github.com/ml-explore/mlx-lm/issues/18) - --steps-per-report controls training report intervals; --steps-per-eval controls validation intervals; --val-batches -1 selects the entire validation iterator instead of a fixed number of batches. on_train_loss_report receives iteration, train_loss, learning_rate, iterations_per_second, tokens_per_second, trained_tokens, and peak_memory. (medium confidence) (verified as of 2026-10-23) — [source](https://raw.githubusercontent.com/ml-explore/mlx-lm/0d4d3d04f8ab7ce4a51e7209e8615755a48ba94d/mlx_lm/lora.py) ## Facts and statements - The current MLX-LM guide accepts YAML through -c/--config, and a repeated command-line flag overrides the corresponding YAML key. Apple's WWDC25 walkthrough says the config file controls learning-rate schedules, optimizer settings, and evaluation intervals. (high confidence) (verified as of 2026-10-23) — [source](https://github.com/ml-explore/mlx-lm/blob/main/mlx_lm/LORA.md) - The guide says an optional valid.jsonl enables periodic validation-loss reporting; --test evaluates test.jsonl and reports test-set perplexity. It also supports sending training metrics to Weights & Biases and/or SwanLab. (low confidence) (verified as of 2026-10-23) — [source](https://github.com/ml-explore/mlx-lm/blob/main/mlx_lm/LORA.md) - Training and validation loss should be paired with a task-level evaluation: an MLX-LM practitioner case study reported validation loss improving while hard-set macro-F1 remained very poor, and a separate local-LLM evaluation guide recommends task-specific measures or human review alongside training metrics. (medium confidence) — [source](https://vincentvandeth.nl/blog/lora-fine-tuning-mlx-lm-qwen35-mac-mini) - For memory-constrained training, the current guide recommends lowering --batch-size from its default of 4 to 2 or 1 at possible speed cost; --grad-accumulation-steps can increase effective batch size without increasing memory, and --grad-checkpoint trades memory for computation. (low confidence) (verified as of 2026-10-23) — [source](https://github.com/ml-explore/mlx-lm/blob/main/mlx_lm/LORA.md) - MLX-LM exposes configuration for iteration count, batch size, validation batches and intervals, loss-report intervals, gradient accumulation, maximum sequence length, learning-rate schedules, and optimizer settings; these controls let a run be adjusted to its training and evaluation needs. (high confidence) (verified as of 2026-10-23) — [source](https://developer.apple.com/videos/play/wwdc2025/298/) - The current trainer reports training loss, the realized learning rate, iterations per second, tokens per second, cumulative trained tokens, and peak memory; validation reports validation loss and elapsed time at the first iteration, configured evaluation intervals, and the final iteration. (high confidence) (verified as of 2026-10-23) — [source](https://github.com/ml-explore/mlx-lm/blob/main/mlx_lm/tuner/trainer.py) - Training or validation loss alone can misstate task quality: in the reported mlx-lm 0.31.3 case, validation loss reached 0.39 while held-out macro-F1 fell to 0.05–0.09 from 0.64–0.78 for the untrained model; a separate BERT fine-tuning study also found different test performance among runs with the same training loss. Pair loss curves with metrics that represent the intended task. (medium confidence) — [source](https://vincentvandeth.nl/blog/lora-fine-tuning-mlx-lm-qwen35-mac-mini) - --learning-rate sets the scalar learning rate. --optimizer accepts adam, adamw, muon, sgd, and adafactor. Put optimizer-specific keyword arguments such as weight_decay under optimizer_config. in YAML; the entry point passes that selected dictionary to the optimizer constructor. (low confidence) (verified as of 2026-10-23) — [source](https://raw.githubusercontent.com/ml-explore/mlx-lm/0d4d3d04f8ab7ce4a51e7209e8615755a48ba94d/mlx_lm/lora.py) - For a schedule, set lr_schedule.name to an mlx.optimizers.schedulers function and lr_schedule.arguments to its positional arguments. An optional warmup count builds a linear schedule from warmup_init (default 0.0) to arguments[0], then joins it to the requested schedule at warmup+1. A truthy lr_schedule takes precedence over the scalar learning_rate. (medium confidence) (verified as of 2026-10-23) — [source](https://raw.githubusercontent.com/ml-explore/mlx-lm/0d4d3d04f8ab7ce4a51e7209e8615755a48ba94d/mlx_lm/lora.py)