v0.7.0

What's Changed

• Optimize Qwen4 n-gram shard gathers by @pyros-projects in #2034
• Fix native quantized KV in Qwen MTP verification by @YuhuaBillChen in #1986
• optize_qwen4exp by @JJJYmmm in #2037
• Support Apodex 1.1 checkpoints by @Lazarus-931 in #2019
• Add MTP support for Qwen3.8-Flash-Next by @lucasnewman in #2040
• Anthropic route: keep in-conversation system messages in place (fixes prefix-cache defeat) by @Lazarus-931 in #2052
• fix(server): transcribe m4a and other non-encodable audio uploads by @Lazarus-931 in #2051
• fix(rfdetr): honor --show-boxes instead of ignoring it by @Anai-Guo in #2024
• fix(sam3): let --threshold fall through to the documented per-task defaults by @Anai-Guo in #2025
• Fix positioned top-k sampling for speculative generation by @tjansn in #2029
• Add continuous batching support for Qwen3.8-Flash-Next by @lucasnewman in #2056
• fix(qwen3.8): support official FP8 conversion by @Thump604 in #2058
• Add APC disk caching for Qwen3.8-Flash-Next by @lucasnewman in #2059
• Fix streamed responses after tool calls by @lucasnewman in #2062
• Fix tool call parsing with quotes and nested values by @lucasnewman in #2064
• fix(attention): materialize a quantized KV state before hand-rolled attention (#2060) by @Anai-Guo in #2065
• Fix nemotron_h prefill crash when inputs and inputs_embeds are both passed by @Lazarus-931 in #2057
• Add pr contributor reminder by @Lazarus-931 in #2068
• Redesign APC for dense and hybrid model caches by @Blaizzy in #1960
• moe-offloading by @Lazarus-931 in #1813
• fix(cache): trim ChunkedKVCache on valid length, not buffer width (#2031) by @Anai-Guo in #2036
• fix(recurrent_gemma): initialize nn.Module in LanguageModel by @Anai-Guo in #2075
• Bump version to 0.7.0rc0 by @Blaizzy in #2083
• perf(qwen3.8): adapt MTP draft depth by @Thump604 in #2046
• feat(qwen3.8): add external PLE storage by @Thump604 in #2045
• handling issue #2073, cache search locally by @Lazarus-931 in #2076
• lfm2_vl: load flattened OptiQ checkpoints (#2088) by @Lazarus-931 in #2092
• fix(video): resolve frame sampling in one place (#2050) by @Lazarus-931 in #2100
• Support float-quantized compressed-tensors checkpoints by @Lazarus-931 in #2102
• diffusion_gemma: strip channel scaffolding from decoded output by @Lazarus-931 in #2101
• fix: require compatible mlx-audio for VoiceChat by @Thump604 in #2110
• fix: stop generation on the tokenizer's EOS when the config omits it by @Lazarus-931 in #2112
• Add LLaVA-OneVision by @Lazarus-931 in #2115
• Fix phantom KV in ragged batched speculative rollback by @Lazarus-931 in #2113
• Add DeepSeek-V4 DSpark speculative drafter by @Lazarus-931 in #2097
• Add LongCat-Flash-Lite-Sparse (LSA + n-gram) by @Lazarus-931 in #2063
• muse_glimmer: load flattened OptiQ checkpoints (#2108) by @Lazarus-931 in #2118
• fix(speculative): detect native MTP drafters via num_nextn_predict_layers by @Lazarus-931 in #2120
• fix(generate): chunk prefill for prompts shorter than prefill_step_size by @Lazarus-931 in #2119
• GLM-5.3 - 2026-08-28 by @Lazarus-931 in #2078
• GLM-5.3 MTP - 2026-08-28 by @Lazarus-931 in #2079
• Fix TurboQuant batch decode, MTP KV quantization, and attention sinks by @YuhuaBillChen in #1822
• Add OmniParser v2 YOLO11 support by @Lazarus-931 in #2122
• fix(florence2): load legacy large-ft checkpoints on transformers>=5.14 by @Lazarus-931 in #2123
• Add Video Depth Anything (video_depth_anything) by @eklipse2k8 in #2104
• Strip Muse Glimmer channel scaffolding from generated output by @Lazarus-931 in #2129
• sam3: fix pred_boxes inflation and offset (RoPE scale, RPB grid, ref_point_head) by @Lazarus-931 in #2128
• Add OpenAI Privacy Filter model support by @lucasnewman in #2131
• Support multiple tool call invocations at once by @lucasnewman in #2132
• Require MLX 0.32.2 for faster speculative decoding by @Blaizzy in #2133
• Add DSpark support for Gemma 4 by @Lazarus-931 in #2137
• qwen3_vl: render per-group video timestamps like the reference processor by @Micha0827 in #2142
• Add DeepSeek-V4 Flash Vision EXP support by @byoan in #2136
• granite4_vision: align deepstack injection with chunked prefill via c… by @Lazarus-931 in #2143
• Register the qwen3_vl_embedding auto-processor patch by @Lazarus-931 in #2146
• Optimize exact Qwen verifier QMV unrolling by @Blaizzy in #2140
• Keep quantized APC checkpoints packed by @Blaizzy in #2090
• Normalize tokenizer padding in BatchGenerator by @Blaizzy in #1951
• Add Nemotron-H speculative decoding support by @Lazarus-931 in #2145
• Enable speculative decoding for Laguna by @Lazarus-931 in #2151
• Fix ArraysCache trim crash on multi-turn reuse for hybrid models by @Lazarus-931 in #2152
• mla: use the absorbed path for multi-token steps by @stooit in #2150
• Fix Gemma4DSparkAttention AttributeError on first forward by @Lazarus-931 in #2158
• Porting model K2-Horizon by @Lazarus-931 in #2153
• Add Ornith 1.5 (Qwen3.5 VL) support and verify qwen3_5_moe MTP by @Lazarus-931 in #2125
• Optimize Qwen4 QSA cache reuse and batched parity by @Blaizzy in #2126
• Fix GLM and Kimi output control tokens by @Lazarus-931 in #2170
• Add MoGe-3 monocular geometry estimation model by @eklipse2k8 in #2171
• Add Z1T-0 (Extropic attention-free sparse LM) by @Lazarus-931 in #2166
• Fix Unlimited-OCR prompt suffix by @Lazarus-931 in #2169
• fix(aya_vision): reference patch_embeddings in the spatial_shapes branch by @Anai-Guo in #2173
• Freeze base model when resuming LoRA training via --adapter-path by @Lazarus-931 in #2181
• fix: declare jinja2 as a runtime dependency by @Shouta0108 in #2176
• Bump version to 0.7.0 by @lucasnewman in #2175

New Contributors

• @pyros-projects made their first contribution in #2034
• @YuhuaBillChen made their first contribution in #1986
• @tjansn made their first contribution in #2029
• @eklipse2k8 made their first contribution in #2104
• @Micha0827 made their first contribution in #2142
• @stooit made their first contribution in #2150
• @Shouta0108 made their first contribution in #2176

Full Changelog: v0.6.17...v0.7.0