v0.8.1

Patch release on the 0.8 line. It adds Xiaomi MiMo v2 Flash and an image-only serving mode, fixes the server exit path so every side-model worker joins before the process ends, and moves the CPU-only CI gates off the GB10 runner. The full record is in CHANGELOG.md.

New Features

• Xiaomi MiMo v2 Flash (model_type mimo_v2_flash): a 48-layer hybrid sliding/full attention sparse decoder with 256 routed experts, separate head widths for Q/K (192) and V (128), per-layer partial RoPE bases, and a noaux_tc sigmoid router with a selection-only correction bias (#2829). mlx-community/MiMo-V2-Flash-4bit loads in about 9 s at 161.8 GiB resident and decodes coherently.
• Image-only serving on both server binaries. --image-model (or MLXCEL_IMAGE_MODEL) with no chat model source now selects an image-only mode instead of failing on a missing --model (#2830). The image worker is the primary model, startup skips chat loading, warmup and the prompt cache, /health follows the image worker, and /v1/models advertises the image id with only the image capability. Chat, completion, tokenizer and prompt-inspection routes answer the structured 501 that names the mode. Combining image-only mode with a router selector, --embedding-model, --reranker-model, --embeddings or --reranking fails at startup naming both flags.

Improvements

• Offline CLI commands install a stderr warn subscriber when RUST_LOG is unset, so WARN and ERROR from generate, inspect, tune, environment fallbacks and deprecated-flag notices are no longer discarded (#2826). stdout is byte-for-byte unchanged.
• MLXCEL_CUDA_DISABLE_GEMV=1 sends gemv-eligible dense matmuls to cuBLASLt as a diagnostic (#2775). gemv stays the default: disabling it cost 21.2% on llama-3.2-1b, 8.9% on llama-3.1-8b and 18.9% on qwen3.5-9b decode throughput on GB10, and cuBLASLt 13.8 resolved only one prefill point against the linked 13.1, a 2.3% TTFT win on the 1B at prompt 4096 that the 9B did not reproduce.

Bug Fixes

• Audio, realtime, embedding, image and rerank workers join before the server exits, so a CPU build no longer aborts with pure virtual method called when an MLX free runs after the C runtime's static destructors start (#2827). Each side provider exposes shutdown_and_wait, the side workers join first and the chat worker second under one deadline, and the remote pipeline stage path shuts its stage down even when the Ctrl-C handler cannot be installed.
• The OpenXLA feature compile job builds again (#2837). safetensors 0.8 returns &str from SafeTensors::names(), and sha2 0.11 returns a digest type with no LowerHex implementation for four format!("{:x}", ...) sites.
• The headed WebKit arm of the WebUI navigation-feedback test allows 150 ms, because Playwright 1.64.1 ships WebKit 27.2 whose two-rAF paint tail measured 74 to 80 ms against 36 to 57 ms on WebKit 26.6, with the application half unchanged at 30 to 34 ms (#2828). Chromium and Firefox keep the 100 ms budget.
• scripts/bench_decode.sh --help matches the script: --max-tokens defaults to 128 rather than 100, --prompt-tokens defaults to 512 and replaces --prompt, and --no-ignore-eos is documented (#2838). The stale-build check runs after argument parsing, so --help works on a binary older than the tree.

CI/CD Improvements

• CPU-only gates moved off the single GB10 runner: clippy and cpu-link run on self-hosted-linux-26.04-x64, and cuda-sm70-compile is disabled because no runner carries the CUDA 12.x toolkit Volta needs. GB10 keeps the four jobs that touch a GPU or the aarch64 release toolchain.
• cargo-deny, overlay-host-tests, webui-contract, webui-bundle and webui-installed-artifact skip changes their inputs cannot affect, the duplicate clippy pass in pipeline-parallel-ci.yml is removed, and the missing filter paths (rust-toolchain.toml, docs/webui/**, the pin-extraction script) are added.
• The startup-notice guard scans every compiled .rs under src/ instead of a fixed file list, resolving calls by module path and enclosing impl type (#2826).

Technical Details

• PROTOCOL_VERSION stays at 5, but cluster membership pins CARGO_PKG_VERSION in discovery beacons and stage assignments, so a v0.8.1 node and a v0.8.0 node cannot share a cluster. Upgrade every node together.
• recipes/registry/0.8.1.json is the architecture registry snapshot for this release, and recipes/registry/CURRENT points at it.
• Benchmark records landed for the ROCm gfx1151 sweep of all 44 checkpoints under models/ (#2774) and for a CSV-only evidence rule that fails on new raw record streams under docs/benchmark_results/data/ (#2836).

Dependencies

None

Breaking Changes

None

Known Issues

• At temperature 0, a block verification path (--prompt-lookup or a draft model) can leave the plain-decode stream, so speculative output is not guaranteed to match plain decode. The root cause is not known (#2778).
• Batched decode still has measured gaps on M1 Ultra: B=1 decode is 3 to 6% lower than before the batched decode work on a few checkpoints, and Mixtral stays below B=1 at B=8 with a warm prompt cache (#2817).
• The ROCm backend remains experimental and validated on gfx1151 only.

What's Changed

• fix(logging): recover CLI warnings and widen the startup guard by @inureyes in #2826
• fix: join side-model workers on server exit by @inureyes in #2827
• fix(webui): calibrate the WebKit navigation feedback budget by @inureyes in #2828
• feat(models): add Xiaomi MiMo v2 Flash support by @inureyes in #2829
• chore(bench): measure cuBLASLt version and gemv A/B on GB10 by @inureyes in #2775
• docs(bench): commit measurement results as CSV, not raw record streams by @inureyes in #2836
• fix(xla): repair the OpenXLA feature compile job after #2819 by @inureyes in #2837
• fix(bench): correct bench_decode.sh help and show it on stale builds by @inureyes in #2838
• docs(benchmarks): add full models/ sweep for ROCm gfx1151 by @inureyes in #2774
• feat: serve image generation without a chat model by @inureyes in #2830
• docs: add technical report for PR #2830 by @inureyes in #2839

Full Changelog: v0.8.0...v0.8.1