rMLX 0.2.8
Qwen3.5-family model-loading correctness and a CI-gateable smoke probe. The
weight-quant loaders no longer corrupt mxfp8/mxfp4 scales, dense Qwen3.5 mxfp8
checkpoints now load via fact-driven dispatch (no longer hardwired to the PARO
path), and rmlx info --probe-smoke returns distinct exit codes so a load
failure can no longer masquerade as success. No breaking changes.
Added
- Dense Qwen3.5 mxfp8 loader + fact-driven dispatch. Both
Qwen3_5ForConditionalGenerationandQwen3_5MoeForConditionalGeneration
now route by checkpoint facts, not the arch string:is_paroquant()selects
the PARO vs the standard loader (the two share an arch string and differ only
byquantization_config.quant_method), a sharedresolve_prefixprobes shard
headers for the tensor prefix, and the MLP block is chosen per layer by tensor
presence (dense SwiGLU vs sparse MoE). A defensive guard hard-errors if a PARO
checkpoint ships MoE expert tensors. Dense Qwen3.5 mxfp8 snapshots now serve
end-to-end. (#191, closes #189)
Fixed
- mxfp8/mxfp4 uint8 E8M0 scales corrupted at load → MoE prefill crash. The
Qwen3.5-MoE and Qwen3 loaders blanket-cast every quantized.scalestensor to
bf16, which is correct for affine (float) scales but corrupts mxfp's uint8 E8M0
scales, crashing the first prefill withdequantize: Scale type must be uint8.
A new per-tensorbf16_scalesgate casts only float scales and passes uint8
scales through verbatim. (#190, closes #188)
Changed
rmlx info --probe-smokenow returns distinct exit codes for CI gating.
Previously every non-Broken*outcome — including a supported-arch load
failure and an inconclusive zero-token run — exited 0, so a loader regression
read as a pass. Exit codes are now0ok,1broken,3load-fail,4
inconclusive,5unsupported (2is reserved for clap arg-parse errors).
healthcheckmarks load-fail / inconclusive / broken as Red and unsupported
as a non-fatal skip. (#193, closes #192)- Bumped
anyhow1.0.102 → 1.0.103 (fixes a Stacked-Borrows UB in
Error::downcast_mut) anduuid1.23.3 → 1.23.4. (#187)