Skip to content

v0.9.0 — LFM2-24B +7% decode, foundry & fusion

Choose a tag to compare

View all tags
@Hmbown Hmbown released this 24 Feb 16:48
· 6 commits to main since this release
v0.9.0
7bc6ac2

Highlights

  • LFM2-24B-A2B-MLX-4bit: +6–7% decode speedup on stock MLX (M4 Max), 500/500 token-identical fidelity
  • D-SIMD gate kernel: fuses softmax+bias+topK into 1 Metal dispatch for D=64 expert gating (2 SIMD groups, 64 threads)
  • Smart K-based defaults: K≤2 (LFM2-8B) → fused SwiGLU +12%; K≥3 (LFM2-24B) → D-SIMD gate +7%. No env vars needed.
  • Foundry module: kernel template evaluation and SFT dataset generation (16 ops, 9 kernel classes)
  • Fusion module: JIT graph tracing and Metal codegen for fused op sequences

Quick Start

pip install -U zmlx
from mlx_lm import load, generate
from zmlx.patch import patch

model, tokenizer = load("LiquidAI/LFM2-24B-A2B-MLX-4bit")
patch(model)  # auto-detects architecture, applies safe fusions
print(generate(model, tokenizer, prompt="Hello!", max_tokens=200))

Stock MLX Benchmarks (M4 Max, 36GB)

Model Baseline Patched Speedup Fidelity
LFM2-8B-A1B-4bit 200.1 tok/s 223.3 tok/s +11.6% 500/500
LFM2-24B-A2B-4bit 152.0 tok/s 161.1 tok/s +6.0% 500/500

Full changelog: https://github.com/Hmbown/ZMLX/blob/main/CHANGELOG.md#090---2026-02-24

Assets 2
Loading