v26.8.11 — Qwen 3.8 Flash Next, MLX 0.32.2

Highlights

• Qwen 3.8 Flash Next runs natively. Alibaba's 125B model with its huge n-gram memory, on Apple silicon. About 60 tok/s on an M4 Max, 78 with speculative decoding, ~70 GB of RAM for the 4-bit pack (ddalcu/Qwen3.8-Flash-Next-MLX-Serve-4bit).
• It sees images and video. Follow-up questions about the same picture answer instantly instead of re-reading it.
• Long prompts stay fast. Sparse attention past 2k tokens runs on custom kernels, so an 8k-token prompt with speculative decoding no longer loses to plain decoding.
• Several chats at once. Concurrent requests on Flash Next share one pass: 2 streams give 1.3x total throughput, 4 streams 1.8x. A single chat is as fast as before.
• Speculative decoding on Flash Next is opt-in (--mtp or the MoE toggle in Settings): +41% on code, a wash on prose, so you choose. It also works on image questions now.
• MLX 0.32.2. Up to +3% faster on MoE models at long context.
• gpt-oss 20B and 120B (OpenAI MoE, harmony format) run natively (#247, thanks @justinluque).

Fixes

• Two concurrent chats on Qwen 3.5 drifted from the single-chat answer.
• Video input on Qwen 3.5/3.8 never reached the model.
• --no-vision still answered image questions; now a clear error.
• --no-mtp was ignored on Flash Next.
• Very long prompts on Flash Next could run out of GPU memory instead of being refused up front.
• A checkpoint the loader cannot read fails that one load instead of taking the server down (#217).
• Shards not named by the model index are no longer loaded or counted toward --max-resident-mem (#274).
• POST /v1/images/edits honours lora_paths / lora_scales (#268).
• A tool call cut off mid-JSON keeps its tool name instead of vanishing from the reply.
• A video whose mp4 encode dropped frames reports an error instead of saving a black clip (#170).
• Thinking models whose generation_config.json declares a thinking default now use it (#219, thanks @Fe2-O3).
• Request image/video/audio buffers no longer leak (#273, thanks @Fe2-O3).
• Unit tests no longer assume non-NAX silicon (#277, thanks @lojza3d).
• --api-key-strict and --api-key-env (#264, thanks @uxsmedjan).
• Video pane: H3 steps and frames reach the ranges the server accepts (#263, thanks @justinluque).
• Music tab: Cover offers the missing fsq.safetensors download on older ACE-Step packs (#276, thanks @Fe2-O3).
• A hybrid (Qwen 3.5/3.8, Nemotron) GGUF served the previous request's tool calls after a long reply: llama.cpp refuses to trim its KV mid-tail, so we cold-prefill instead (#286, #287, thanks @twotonetobi).
• Tool-call arguments keep their own whitespace: an old_string with leading indentation no longer loses it, so edits land at the right nesting (#294, thanks @Agnik47).
• A cancelled prefill on a hybrid model now keeps the prefix it already computed, so the retry resumes instead of starting over (#270, thanks @codysk).
• A chained 5-window H3 video no longer renders for half a minute and then fails to deliver; over-cap requests are refused up front (#283, thanks @ClackShen).
• Image edits with an explicit output size get exactly that size (#290, thanks @justinluque).
• The max resident models setting is exposed in Settings (#289, thanks @justinluque).
• Stopping the server resets every loaded model so the model picker pill is correct (#291, thanks @justinluque).
• Chat: a generated image is drawn from its file instead of a second copy kept in the history (#293, thanks @lojza3d).
• seed now replays a sampled reply byte for byte. It was only honoured with logprobs on, and even then every token drew from the same key.

Thank you for all the contributions !