Skip to content

v0.8.0 — GLM/Qwen3 MoE decode + exo integration

Choose a tag to compare

@Hmbown Hmbown released this 04 Feb 16:44
· 74 commits to main since this release

What's new

  • GLM-4.7-Flash decode +8% — 46 MoE layers fused via gather_qmm_swiglu custom primitive
  • Qwen3-30B-A3B decode +6% — same fused MoE path
  • exo integration — one-command setup (bash setup_zmlx.sh) for running ZMLX-accelerated models in exo clusters. See docs/EXO.md.
  • mlx-lm compatibility layer — handles API differences across mlx-lm versions
  • Auto-skip safety — models auto-skip on stock MLX when custom primitive is unavailable (0% change, no regressions)

Requirements

  • Stock MLX: LFM2 gains work out of the box
  • GLM/Qwen3 gains: requires building the custom gather_qmm_swiglu primitive (see docs/EXPERIMENTAL_MLX.md)

Benchmarks

Model Hardware Change
LFM2-8B-A1B-4bit M4 Max 36 GB +11.6%
GLM-4.7-Flash-4bit M4 Max 36 GB +8.1%
GLM-4.7-Flash-4bit M4 Mac Studio +8%
Qwen3-30B-A3B-4bit M4 Max 36 GB +5.5%

All results token-identical under greedy decoding.