v0.8.0 — GLM/Qwen3 MoE decode + exo integration
What's new
- GLM-4.7-Flash decode +8% — 46 MoE layers fused via
gather_qmm_swiglucustom primitive - Qwen3-30B-A3B decode +6% — same fused MoE path
- exo integration — one-command setup (
bash setup_zmlx.sh) for running ZMLX-accelerated models in exo clusters. See docs/EXO.md. - mlx-lm compatibility layer — handles API differences across mlx-lm versions
- Auto-skip safety — models auto-skip on stock MLX when custom primitive is unavailable (0% change, no regressions)
Requirements
- Stock MLX: LFM2 gains work out of the box
- GLM/Qwen3 gains: requires building the custom
gather_qmm_swigluprimitive (see docs/EXPERIMENTAL_MLX.md)
Benchmarks
| Model | Hardware | Change |
|---|---|---|
| LFM2-8B-A1B-4bit | M4 Max 36 GB | +11.6% |
| GLM-4.7-Flash-4bit | M4 Max 36 GB | +8.1% |
| GLM-4.7-Flash-4bit | M4 Mac Studio | +8% |
| Qwen3-30B-A3B-4bit | M4 Max 36 GB | +5.5% |
All results token-identical under greedy decoding.