vMLX 1.6.34

vMLX 1.6.34

Performance

• Native MTP overhaul (Qwen3.8, dots3-note): aligned draft-head context cache plus generalized skip-replay — rejected drafts no longer replay through the main model at any depth, and restore-aware acceptance gates keep heads warm across cache restores. Warm in-app decode on Qwen3.8-27B rises from ~22 to 38–48 t/s; dots3-note reaches ~41 t/s.
• DFlash2 session prefix reuse: end-of-turn cache checkpoints, draft hidden-state gap splice, and prompt-boundary snapshots — warm multiturn TTFT drops ~20x; 60+ t/s sustained on Qwen3.8 with DFlash2 (68.7 t/s measured). The DFlash runtime now ships in the bundle.
• Stream interval defaults to 8: the renderer can no longer backpressure-stall the engine emit loop on long conversations; legacy sessions are lifted automatically.
• dots3-note deep context: per-chunk prefill admission projection (removes a phantom ~17k refusal), companion snapshot exemption for positional latents (quadratic memory growth fixed), and a bounded L2 pending-write budget. 42–56k-token conversations verified in-app on a 128GB Mac with the wired limit raised.
• mlx 0.32.1, mflux 0.19.0.

Stability

• Fixed a store-path memory explosion that could take down the whole machine. The prefix-cache block store imported numpy slice views into MLX in a way that dragged a full-layer-sized buffer per block (~140GB of live Metal on an 11k-token hybrid store). At the default wired limit this silently killed the engine minutes after a turn; with a raised limit it could wedge the machine into a watchdog kernel panic. Slice imports are now slice-sized, background clean prefills return their chunk transients per chunk, and the idle re-derive no longer duplicates the store pass.
• Fixed a transient DFlash2 resume crash ([full] Negative dimensions not allowed) on short follow-up turns in resumed conversations.
• Removed the RAM preflight that refused large model loads — estimates now advise, never refuse; a 101GB dots3-note bundle loads again on a 128GB Mac.
• Native MTP actually engages by default — bundle-temperature auto-detection had kept it permanently off; detection is now compatible-only with a deterministic-defaults sampling policy.

Fixed

• Qwen3.5-family video input works — videos route as sampled frames and temporal questions answer correctly (previously failed with a feature/token mismatch).
• Prefill admission rejections include a wired-limit advisory naming the exact sysctl, the macOS ~84%-of-RAM default, and the reset-on-reboot behavior.
• Hybrid prefix cache: bf16 cumulative-state dtype round-trip; plain-KV restores step-padded.