v0.7.0rc0

What's Changed

• Optimize Qwen4 n-gram shard gathers by @pyros-projects in #2034
• Fix native quantized KV in Qwen MTP verification by @YuhuaBillChen in #1986
• optize_qwen4exp by @JJJYmmm in #2037
• Support Apodex 1.1 checkpoints by @Lazarus-931 in #2019
• Add MTP support for Qwen3.8-Flash-Next by @lucasnewman in #2040
• Anthropic route: keep in-conversation system messages in place (fixes prefix-cache defeat) by @Lazarus-931 in #2052
• fix(server): transcribe m4a and other non-encodable audio uploads by @Lazarus-931 in #2051
• fix(rfdetr): honor --show-boxes instead of ignoring it by @Anai-Guo in #2024
• fix(sam3): let --threshold fall through to the documented per-task defaults by @Anai-Guo in #2025
• Fix positioned top-k sampling for speculative generation by @tjansn in #2029
• Add continuous batching support for Qwen3.8-Flash-Next by @lucasnewman in #2056
• fix(qwen3.8): support official FP8 conversion by @Thump604 in #2058
• Add APC disk caching for Qwen3.8-Flash-Next by @lucasnewman in #2059
• Fix streamed responses after tool calls by @lucasnewman in #2062
• Fix tool call parsing with quotes and nested values by @lucasnewman in #2064
• fix(attention): materialize a quantized KV state before hand-rolled attention (#2060) by @Anai-Guo in #2065
• Fix nemotron_h prefill crash when inputs and inputs_embeds are both passed by @Lazarus-931 in #2057
• Add pr contributor reminder by @Lazarus-931 in #2068
• Redesign APC for dense and hybrid model caches by @Blaizzy in #1960
• moe-offloading by @Lazarus-931 in #1813
• fix(cache): trim ChunkedKVCache on valid length, not buffer width (#2031) by @Anai-Guo in #2036
• fix(recurrent_gemma): initialize nn.Module in LanguageModel by @Anai-Guo in #2075
• Bump version to 0.7.0rc0 by @Blaizzy in #2083
• perf(qwen3.8): adapt MTP draft depth by @Thump604 in #2046
• feat(qwen3.8): add external PLE storage by @Thump604 in #2045
• handling issue #2073, cache search locally by @Lazarus-931 in #2076
• lfm2_vl: load flattened OptiQ checkpoints (#2088) by @Lazarus-931 in #2092
• fix(video): resolve frame sampling in one place (#2050) by @Lazarus-931 in #2100
• Support float-quantized compressed-tensors checkpoints by @Lazarus-931 in #2102
• diffusion_gemma: strip channel scaffolding from decoded output by @Lazarus-931 in #2101
• fix: require compatible mlx-audio for VoiceChat by @Thump604 in #2110
• fix: stop generation on the tokenizer's EOS when the config omits it by @Lazarus-931 in #2112
• Add LLaVA-OneVision by @Lazarus-931 in #2115
• Fix phantom KV in ragged batched speculative rollback by @Lazarus-931 in #2113
• Add DeepSeek-V4 DSpark speculative drafter by @Lazarus-931 in #2097
• Add LongCat-Flash-Lite-Sparse (LSA + n-gram) by @Lazarus-931 in #2063
• muse_glimmer: load flattened OptiQ checkpoints (#2108) by @Lazarus-931 in #2118
• fix(speculative): detect native MTP drafters via num_nextn_predict_layers by @Lazarus-931 in #2120
• fix(generate): chunk prefill for prompts shorter than prefill_step_size by @Lazarus-931 in #2119
• GLM-5.3 - 2026-08-28 by @Lazarus-931 in #2078

New Contributors

• @pyros-projects made their first contribution in #2034
• @YuhuaBillChen made their first contribution in #1986
• @tjansn made their first contribution in #2029

Full Changelog: v0.6.17...v0.7.0rc0