-
Notifications
You must be signed in to change notification settings - Fork 0
dflash mlx trial.stub
id: dflash-mlx-trial name: DFlash × MLX repo_path: ~/Documents/sw30labs/repos/dflash-mlx-trial remote_url: https://github.com/sw30labs/dflash-mlx-trial.git primary_language: Python framework: MLX ingested: 2026-07-04 last_commit_date: 2026-06-18 category: local-inference-mlx stacks: [mlx]
A trial harness that points DFlash's block-diffusion speculative decoding at Qwen3.6-27B, running natively on Apple Silicon via MLX and measuring the speedup. DFlash drafts a whole block of tokens in parallel; the full model verifies them in one forward pass; the matching prefix is kept — so output is byte-for-byte identical to plain decoding, just faster because more tokens land per verify step.
-
scripts/setup.sh— venv + deps, no model downloads. -
scripts/run.sh— runs the target/draft pair (--target mlx-community/Qwen3.6-27B-bf16 --draft z-lab/Qwen3.6-27B-DFlash). -
examples/mlx_smoke_test.py— the measured smoke test. - Documents serving the same pair through oMLX behind an OpenAI-compatible API.
On an Apple M3 Ultra (512 GB), full-precision Qwen3.6-27B-bf16 target + Qwen3.6-27B-DFlash drafter, 512 tokens greedy: throughput 12.6 → 42.3 tok/s (≈3.37× speedup), mean 7.33 tokens accepted per verify step, 57.7 GB peak. The bigger/slower the target, the more parallel drafting pays off (a smaller Qwen3.5-4B pair hit only ~2.0×).
- mlx / mlx-lm
- DFlash (z-lab/dflash, MLX backend)
- oMLX (optional — OpenAI-compatible local serving)
None (trial/benchmark repo; shell-driven).
mlx apple-silicon local-inference qwen benchmark python