v0.8.4

Highlights

• MoE (Qwen, experimental): new env-gated routing kernel path via router_argpartition_logits_topk().
  • Enable with:
    • ZMLX_QWEN_ROUTER_ARGPARTITION_LOGITS=1
    • ZMLX_QWEN_ROUTER_ARGPARTITION_LOGITS_TOPK=1 (requires the above)
  • Off by default; intended for controlled benchmarks.
• Bench evidence: v8 promoted stack and v9 reproduction/routertopk suites are committed under benchmarks/repro_capsules/ with matching benchmarks/matrix.jsonl rows.
• Custom MLX context (optional): build mx.gather_qmm_swiglu locally via integrations/mlx_local_integration/ (no changes to upstream MLX required until you opt in).
• exo usage: run exo with runtime patching via zmlx-exo (see docs/EXO.md).

Install / Use

• Install ZMLX: pip install zmlx
• Run with exo (in the same env where exo is installed): zmlx-exo
• Optional custom MLX primitive (for GLM/Qwen3 gains): bash integrations/mlx_local_integration/setup_mlx_local.sh (see docs/EXPERIMENTAL_MLX.md)

Repro Capsules (Source of Truth)

Promoted v8 (summary capsules):

• benchmarks/repro_capsules/qwen3_a3b_combo_v8_fp32nofmaonly_t200_r2_summary.json
• benchmarks/repro_capsules/qwen3_a3b_combo_v8_fp32nofmaonly_t1024_r2_summary.json
• benchmarks/repro_capsules/glm47_combo_v8_fp32nofmaonly_t200_r2_summary.json
• benchmarks/repro_capsules/glm47_combo_v8_fp32nofmaonly_t1024_r2_summary.json

v9 reproduction + routertopk (summary capsules):

• benchmarks/repro_capsules/qwen3_combo_v9_repro_t200_r3_summary.json
• benchmarks/repro_capsules/qwen3_combo_v9_repro_t1024_r2_summary.json
• benchmarks/repro_capsules/glm47_combo_v9_repro_t200_r3_summary.json
• benchmarks/repro_capsules/glm47_combo_v9_repro_t1024_r2_summary.json
• benchmarks/repro_capsules/qwen3_combo_v9_routertopk_t200_r3_summary.json
• benchmarks/repro_capsules/qwen3_combo_v9_routertopk_t1024_r2_summary.json
• benchmarks/repro_capsules/qwen3_combo_v9_routertopk_t1024_r3_confirm_summary.json

Notes

• The router top-k kernel path is experimental and not promoted as a default.