Ishizuki v0.6.0 — DeepSeek-V4.1, DFlash drafting, OrcaSAQ2

DeepSeek-V4.1 from its release shards, DFlash drafting, and OrcaSAQ2

DFlash 2 drafting doubles code and math

• Qwen3.8 packs now draft in blocks with z-lab's DFlash 2 drafter. The target model verifies the whole block through a new matrix-fragment kernel, so checking eight rows costs about 2.6 plain steps instead of more than five on MLX's own path.
• Greedy and sampled output both work. Sampled drafting keeps the same distribution as plain sampling.

Measured on an idle M1 Max, greedy, 256 tokens:

Pack	Plain	DFlash (math / code)
Ternary Bonsai 2 27B, 2-bit	22.5 tok/s	45.0 / 45.0 tok/s
Qwen3.8-27B EXL3 2.0 bpw	9.2 tok/s	28.7 / 35.4 tok/s
OrcaSAQ2 27B	8.3 tok/s	28.9 / 33.3 tok/s
Qwen3.8-27B IQ2_XS GGUF	10.6 tok/s	23.9 / 24.7 tok/s

On the same machine and the same Bonsai pack, mlx-lm decodes at 20.9 tok/s and llama.cpp runs the GGUF at 12.8 tok/s. Prose gains little from drafting, because the drafter guesses prose far less often. BENCHMARKS.md has the full side-by-side table.

DeepSeek-V4.1-Flash runs from its release shards

• The 48 shards load as they ship, with no conversion step. The fp8 matrices run as MLX mxfp8 and the fp4 experts as mxfp4, byte for byte. The experts stream from disk into as many slots as the machine holds, and the two 100 GB n-gram tables are read a row at a time.
• Every part of the architecture was written against DeepSeek's reference code and checked against it, on a small fixture and on real layers.
• Streamed reads no longer fill the file cache. macOS only skips the cache when a read is aligned to its 16 KB pages, which safetensors offsets almost never are. Reads now go through page-aligned scratch memory, so a long prompt no longer squeezes other apps into compressed memory. Cold decode improved from 0.50 to 0.66 tok/s.
• Off the internal SSD on an M1 Max: 14.2 tok/s prefill and 1.29 tok/s decode, warm.

Load more of what's out there

• OrcaSAQ2 packs load. They are EXL3 with an int8 embedding that has one scale per row, and they answer correctly.

Streamed experts, faster

• Expert slots are now locked in memory. Before, macOS compressed idle slots, and a later hit waited on decompression. On the 125B-A6B the step after a 4k prefill went from 4.55 s to 1.71 s.
• Prefill reads each expert once per chunk, not once per token. 24.6 → 37.5 tok/s on 1k tokens, and 65.8 tok/s on 4k tokens in 2k-token chunks.
• Slot count defaults to Automatic, sized to what fits on the machine.
• Politeness now defaults to Normal. Adaptive, which backs off for heat, cut prefill chunks to a quarter of their size; it's still one setting away.

The app

• A new look: parchment and paper cards, with warm charcoal after dark, moss as the accent colour and New York as the typeface. All text is one size larger.
• The agent can be steered mid-turn, and can stop to ask you a question, with its options shown as buttons above the composer.
• A conversation keeps the model it started with. Opening a conversation preloads it into the cache so its next turn starts sooner.
• Apple Intelligence is hidden when its model files are missing.

For developers

• EngineBenchProbe benches any pack plain, with MTP and with DFlash (ISHIZUKI_BENCH_PACK; set ISHIZUKI_BENCH_SHOW to print what the pack said).
• DeepSeekReleaseProbe and DeepSeekStepProbe time DeepSeek-V4.1 off a drive. They run only when ISHIZUKI_DEEPSEEK is set.
• DFlashProbe compares a drafted run against a saved plain run.

Full Changelog: v0.4.0...v0.6.0