v0.7.12

What's Changed

• Lower MOSS-Image peak memory during VAE decode by @felix-ab in #2615
• Fix LFM3 thinking leakage by initializing response parsers with the prompt by @Blaizzy in #2621
• Add Raon-1 Mini support by @Blaizzy in #2622
• Support text-only ASR with diarized_json by @Blaizzy in #2626
• Send SSE heartbeats during idle OpenAI streams by @Blaizzy in #2633
• Add native Mage-VL-Cowork support by @Blaizzy in #2636
• Support speculative thinking budgets and expose reasoning efforts by @Blaizzy in #2641
• Add Mistral Large 4 by @Blaizzy in #2629
• fix: accept top_k equal to vocab size in apply_top_k by @ShryukGrandhi in #2643
• Make the LFM3-VL processor follow the transformers processor contract by @abhijithneilabraham in #2640
• Expose the server's reasoning-effort control in the generate CLI by @selimfedakar in #2631
• Filter per-row M-RoPE deltas when the speculative batch drops finished rows by @selimfedakar in #2625
• Keep tool-call streaming alive on outputs the final parser tolerates by @selimfedakar in #2635
• convert: keep the source tokenizer.json after save_pretrained by @JarJarFran in #2624
• fix(flint): view flat uint8 quantized weights as packed uint32 by @martinstiago in #2616
• Fix Gemma 4 tool calls containing braces in strings by @victor-torres in #2593
• fix: reload native LFM2 ColBERT checkpoints by @kocean9-freedom in #2575
• fix(tool-parsers): keep the text "null" for string-typed qwen3_coder parameters by @parasew in #2588
• Preserve inference state when resuming interrupted responses by @Blaizzy in #2623
• Honor APC memory admission and keep warm requests decoding by @Blaizzy in #2645
• Reduce APC merge memory and preserve custom checkpoints by @Blaizzy in #2646
• Fix typical-p filtering with masked tokens by @HowieYin in #2563
• fix: honor base64 embedding response encoding by @kocean9-freedom in #2573
• Build the Qwen3 text reranker prompt without the chat template by @begcdn in #2561
• fix: antialias position embedding resizes with PyTorch's triangle filter by @rrrxxx0510 in #2553
• Honor configured activations in BERT and XLM-R encoders by @kocean9-freedom in #2572
• Expose continuation idle timeout to clients by @Blaizzy in #2652
• Reduce continuation idle timeout to two minutes by @Blaizzy in #2654
• Add native Idol image-to-3D generation by @Blaizzy in #2644
• fix(qwen3_vl): compute text-only rope deltas from unpadded length by @pujitha24 in #2587
• Keep resumed DFlash and MTP requests in shared speculative batches by @Blaizzy in #2648
• fix: compute text-only rope deltas from unpadded length in the other MRoPE models by @ahuzmeza in #2658
• qwen3_5: apply YaRN from rope_parameters on the VLM rotary by @xocialize in #2649
• fix: preserve the input value domain through video frame resizing by @rrrxxx0510 in #2551
• fix: interpolate position embeddings over the spatial dimensions by @rrrxxx0510 in #2549
• Support resumable diffusion generation at block boundaries by @Blaizzy in #2662
• Improve Mistral DSpark decode stability and speed by @Blaizzy in #2659
• fix(prompt_utils): keep audio after image for media-only LFM3 turns by @selimfedakar in #2655
• Bound prefill work and publish decoded tokens immediately by @Blaizzy in #1630
• Generalize trailing-logit projection and greedy selection by @Blaizzy in #2664
• docs: update MLX-VLM branding for Neywa Labs by @Blaizzy in #2666
• Document video= and fps= on apply_chat_template and warn on num_videos by @AkshayShah03 in #2479
• Honour max_completion_tokens on /v1/chat/completions by @AkshayShah03 in #2484
• Number phi3_v/phi4mm image tokens across the whole conversation by @AkshayShah03 in #2492
• Fix PaddleOCR-VL LoRA training: freeze visual encoder, accept packed patches by @AkshayShah03 in #2476
• Cut streamed /v1/messages text at stop_sequences by @AkshayShah03 in #2486
• Fix Eagle3DraftModel.bind for drafters without embed_tokens by @AkshayShah03 in #2473
• fix: update project references after Neywa Labs transfer by @Blaizzy in #2667
• feat: serve d1-2B and d1-omni-2B on /v1/decisions by @ykhrustalev in #2657
• Bump version to 0.7.12 by @lucasnewman in #2663

New Contributors

• @felix-ab made their first contribution in #2615
• @ShryukGrandhi made their first contribution in #2643
• @abhijithneilabraham made their first contribution in #2640
• @selimfedakar made their first contribution in #2631
• @JarJarFran made their first contribution in #2624
• @martinstiago made their first contribution in #2616
• @victor-torres made their first contribution in #2593
• @kocean9-freedom made their first contribution in #2575
• @parasew made their first contribution in #2588
• @HowieYin made their first contribution in #2563
• @begcdn made their first contribution in #2561
• @rrrxxx0510 made their first contribution in #2553
• @pujitha24 made their first contribution in #2587
• @ahuzmeza made their first contribution in #2658
• @AkshayShah03 made their first contribution in #2479

Full Changelog: v0.7.11...v0.7.12

v0.7.11

What's Changed

• feat(server): live-input STT WebSocket on the audio worker by @TheAngryPit in #2167
• Add SoMark OXR-1.0 document parsing model by @bxespinozar in #2560
• Reduce CI turnaround with sharding and cheaper parity tests by @Blaizzy in #2567
• fix: keep converted metadata writable by @ykhrustalev in #2298
• fix: record converted dtype in exported configs by @Blaizzy in #2577
• Fix text-only input truncation by @Mengmj in #2552
• feat: native Nemotron 3.5 Parse MLX port by @geoHeil in #2562
• Fix ModelOpt NVFP4 AWQ checkpoint loading and input scaling by @Blaizzy in #2582
• Expose generation timings in Responses API results by @Blaizzy in #2583
• perf: make d1 and d1-omni faster and use less memory by @ykhrustalev in #2500
• Add MiniMax M3.1 Flash support by @Blaizzy in #2580
• (feat) Add mtp to trainers by @Goekdeniz-Guelmez in #2558
• Add streamed diarized transcriptions and speaker analysis endpoints by @Blaizzy in #2600
• Bump version to 0.7.11 by @lucasnewman in #2599
• docs: list Granite Vision 4.1 and Mistral-Small-4 in model READMEs by @DineshKadiam in #2598
• Recover compaction failures and reuse chat prefixes by @Blaizzy in #2602
• Add Unlimited-OCR-2 support (page2d RoPE + DASH sparse attention) by @shuuul in #2601
• Add native LFM3 and LFM3-VL support by @Blaizzy in #2607
• Align LFM3 video and reasoning with release checkpoints by @Blaizzy in #2608
• Add LFM3-Audio support by @Blaizzy in #2609
• Add LFM3-Omni support by @Blaizzy in #2610
• Add LFM3 d1 decision models by @Blaizzy in #2611
• Add MiniCPM-OCR 1.0 and Jev support by @Blaizzy in #2606
• Fix Unlimited-OCR FP32 fusion and chunked prefill reference retention by @bearcatty in #2603
• Forward native STT prompts and task selection in the audio API by @Blaizzy in #2618
• Exclude transient request context from compaction history by @Blaizzy in #2612
• Exclude tool-call examples in reasoning from executable calls by @Blaizzy in #2613
• Recover long tool conversations and reduce cache overhead by @Blaizzy in #2617

New Contributors

• @TheAngryPit made their first contribution in #2167
• @bxespinozar made their first contribution in #2560
• @Mengmj made their first contribution in #2552
• @geoHeil made their first contribution in #2562
• @DineshKadiam made their first contribution in #2598
• @bearcatty made their first contribution in #2603

Full Changelog: v0.7.10...v0.7.11

v0.7.10

What's Changed

• Add Muse Spark Open support with audio/video/images by @lucasnewman in #2542
• Add native Flint BF16 support and bump to 0.7.10 by @Blaizzy in #2565
• Preserve NVFP4 scales and batch offloaded MoE execution by @Blaizzy in #2538
• Fix fused NVFP4 rowwise scales in expert offload repacking by @Blaizzy in #2566

Full Changelog: v0.7.9...v0.7.10

v0.7.9

What's Changed

• Fix MiniCPM vision dtype with quantized embeddings by @shiftedx in #2511
• Simplify MiniCPM-V 4.7 vision cache handling by @Blaizzy in #2514
• qwen3_5: read strided K/V in ragged decode attention instead of copying the KV cache every step by @griffinwork40 in #2510
• Fix Mistral Large 3 image feature merging by @Blaizzy in #2516
• Add Nemotron 3.5 Super VL with MTP and Bolt support by @Blaizzy in #2528
• Add live image resolution controls to the server by @Blaizzy in #2529
• Release prompt cache of finished SpeculativeGenerationBatch by @sady4850 in #2525
• Read PIL images as channels-last in the Gemma 4 image processors by @MohammadHijjawi97 in #2367
• sam3: use PyTorch channel grouping in the pixel decoder GroupNorm by @ju-ius in #2513
• Read PIL images as channels-last in the EmbeddingGemma 2 processors by @zviratko in #2489
• Add grpo for vlms by @Goekdeniz-Guelmez in #2504
• Fix remaining affine QMV precision mismatches on MLX 0.32.4 by @griffinwork40 in #2527
• Add support for Flux-3 image generation and editing by @lucasnewman in #2537
• Add MOSS-Image-SFT image generation model by @lucasnewman in #2541
• Honor checkpoint sampling defaults in CLI and generation APIs by @Blaizzy in #2545
• Fix rotating cache sinks and absolute position handling by @Blaizzy in #2548
• Bump version to 0.7.9 by @lucasnewman in #2555

New Contributors

• @shiftedx made their first contribution in #2511
• @griffinwork40 made their first contribution in #2510
• @sady4850 made their first contribution in #2525
• @MohammadHijjawi97 made their first contribution in #2367
• @ju-ius made their first contribution in #2513
• @zviratko made their first contribution in #2489

Full Changelog: v0.7.8...v0.7.9

v0.7.8

What's Changed

• adding kolibri1 support by @Lazarus-931 in #2423
• Add EmbeddingGemma 2 multimodal embedding support by @Blaizzy in #2446
• qwen3_5: use mlx-lm's packed gated delta kernel for prefill by @wyanzhao in #2442
• Use fused SDPA in the RF-DETR backbone attention by @ishaanzee in #2443
• Compute grid_sample in float32 for half-precision inputs by @ishaanzee in #2454
• Fix RF-DETR small variant config by @ishaanzee in #2453
• Make RF-DETR sanitize idempotent by @ishaanzee in #2452
• Fix off-by-one RF-DETR backbone feature layers by @ishaanzee in #2450
• Fix tracker RoPE axis order to match Meta checkpoints by @atesahmet0 in #2435
• Add support for Clef decision model by @Lazarus-931 in #2459
• Support Xing4.0 model by @Lazarus-931 in #2465
• feat: add LiquidAI d1-3B and d1-omni-600M decision models by @ykhrustalev in #2467
• inkling: use the Metal kernels only when Metal is available by @joshuaswarren in #2462
• Use the separable interpolation kernel for the RF-DETR segmentation u… by @ishaanzee in #2468
• Fix/2426 expert activation by @Lazarus-931 in #2449
• moe_offload: write a weight index describing the repacked output by @Lazarus-931 in #2427
• Migrate to httpx2, fix CI by @lucasnewman in #2487
• fix: recognize LFM2's down projection in mixed quant recipes by @ykhrustalev in #2497
• Preserve NVFP4 global scales by @Blaizzy in #2437
• Load bundled drafters directly in CLI and server by @Blaizzy in #2457
• Stabilize tool-schema key order for prefix caching by @Blaizzy in #2498
• Add MiniCPM-V 4.7 1B and 35B-A3B with a custom processor by @Blaizzy in #2499
• Reformat trainer for all modalities by @Goekdeniz-Guelmez in #1939
• Fix Muse Glimmer ignoring explicit RoPE scaling settings by @spector-in-london in #2494
• Fix cancellation during multi-row prefill by @Blaizzy in #2273
• Fix compaction when max output fills the context window by @Blaizzy in #2458
• Stream tool arguments in Chat Completions and Responses by @Blaizzy in #2502
• Allow compaction between completed tool groups by @Blaizzy in #2501

New Contributors

• @wyanzhao made their first contribution in #2442
• @ishaanzee made their first contribution in #2443
• @atesahmet0 made their first contribution in #2435
• @joshuaswarren made their first contribution in #2462
• @spector-in-london made their first contribution in #2494

Full Changelog: v0.7.6...v0.7.8

v0.7.6

What's Changed

• Stream compaction progress and recover oversized requests by @Blaizzy in #2418
• Bump version to 0.7.6 by @lucasnewman in #2436

Full Changelog: v0.7.5...v0.7.6

v0.7.5

What's Changed

• Enable Qwen3-Omni multimodal batching by @Lazarus-931 in #2349
• Review/extraction test harness by @Lazarus-931 in #2394
• Fix broken CI: Qwen-compatible tokenizer in the processor regression test by @Lazarus-931 in #2401
• Add Decider and the shared typed decision API by @Lazarus-931 in #2396
• Add Laya on the shared decision API by @Lazarus-931 in #2397
• Keep the most likely MoG component for a tiny top_p in Nemotron VoiceChat by @pierre427 in #2405
• Stop gradients on MoE router selection indices (MLX >= 0.32.1) by @Lazarus-931 in #2409
• removing duplicate rotate_half and check_array_shape funcs by @Lazarus-931 in #2414
• Feat/decision cli by @Lazarus-931 in #2402
• Add conversation compaction to the Responses API by @Blaizzy in #2408
• Fix prequantized Qwen4 PLE loading by @Lazarus-931 in #2417
• Fix PhiMoE routing and plain RoPE configs by @Lazarus-931 in #2419
• Compute top-p and typical-p probability mass in float32 by @pierre427 in #2404
• qwen3_vl: take video timestamps from the frames that were actually sampled by @Marian2110 in #2413
• Bump version to 0.7.5 by @lucasnewman in #2432

New Contributors

• @Marian2110 made their first contribution in #2413

Full Changelog: v0.7.4...v0.7.5

v0.7.4

What's Changed

• Generate multiple images per prompt via num_images (ming_image) by @Lazarus-931 in #2353
• Feat/z image num images by @Lazarus-931 in #2354
• Fix MiMo-V2.6 audio requests failing across threads in the server by @Sofille65 in #2352
• Batched num_images generation for qwen_image, flux2, ideogram4 by @Lazarus-931 in #2355
• Fix Qwen3.5-arch left padding for batches by @lucasnewman in #2357
• Feat/qwen3 5 moe text by @Lazarus-931 in #2340
• gpt-oss: parse harmony channels into reasoning and content by @Lazarus-931 in #2329
• Fix the packed width of empty 3-, 5- and 6-bit quantized KV caches by @pierre427 in #2342
• Fix small top-p and very low temperature sampling by @lucasnewman in #2358
• Preserve interleaved image order in shared prompt formatting by @Blaizzy in #2362
• Fix Qwen3-VL position embeddings with a quantized pos_embed by @rcfa in #2304
• Fix three tool-parser failures: boolean property schemas, untyped Qwen3-Coder objects, unclosed parameters by @pierre427 in #2344
• Feat/lfm2 colbert by @Lazarus-931 in #2360
• Cleanup/model test ownership by @Lazarus-931 in #2364
• Refactor/gliner model scope by @Lazarus-931 in #2363
• Cleanup/moondream qwen test ownership by @Lazarus-931 in #2366
• tests: remove restored Mistral test class by @Lazarus-931 in #2371
• docs: link corrected multimodal checkpoints by @Lazarus-931 in #2372
• Strip newlines after the thinking close tag across stream chunks by @pierre427 in #2369
• Use model-based sampling defaults for image generation by @lucasnewman in #2373
• Make streamed tool-call content match the non-streamed response by @pierre427 in #2365
• Feat/lfm2 encoder by @Lazarus-931 in #2359
• Return CacheList from Florence-2 and Nemotron-Parse make_cache by @Lazarus-931 in #2381
• Fix Gemma 4 audio encoder attending one frame too far into the past by @yaanfpv in #2385
• Support/internvl3 5 by @Lazarus-931 in #2375
• DeepSeek V4.1 vision and language model support by @Lazarus-931 in #2214
• Bump version to 0.7.4 by @lucasnewman in #2388

New Contributors

• @Sofille65 made their first contribution in #2352
• @pierre427 made their first contribution in #2342
• @rcfa made their first contribution in #2304
• @yaanfpv made their first contribution in #2385

Full Changelog: v0.7.3...v0.7.4

v0.7.3

What's Changed

• Hemmingway-1 support by @Lazarus-931 in #2330
• MiMo-V2.6-Flash-RL support by @Lazarus-931 in #2327
• limite-1b-violetto support by @Lazarus-931 in #2331
• Qwen-Image-2.1 fixes by @lucasnewman in #2332
• Add kv caching for Qwen-Image-2.1 by @lucasnewman in #2333
• Fix MiMo-V2.6 native MXFP4 loading by @Lazarus-931 in #2337
• Providing MiMo-V2.6 multimodal support by @Lazarus-931 in #2338
• Ming-Image-0.1-Design (text-to-image) support by @Lazarus-931 in #2334
• Release generation batch arrays without waiting for cyclic GC by @aprylewu in #2328
• Include batching support for Mimo v2.6 e by @Lazarus-931 in #2339
• Add rampart support and unify pii model's shared components by @Lazarus-931 in #2341
• Enable batched audio for supported processors by @Lazarus-931 in #2348
• Enable Gemma-3n audio batching by @Lazarus-931 in #2350
• Bump version to 0.7.3 by @lucasnewman in #2351

New Contributors

• @aprylewu made their first contribution in #2328

Full Changelog: v0.7.2...v0.7.3

v0.7.2

What's Changed

• Downmix stereo audio before resampling in load_audio by @kiarina in #2258
• Fix APC prefill and restore memory planning by @Blaizzy in #2262
• Move APC cache layout logic into cache classes by @Blaizzy in #2263
• fix: correct LFM2-VL/LFM2.5-VL vision tower and preprocessing by @ykhrustalev in #2266
• deepseek_v4: restore five weights dropped during checkpoint key normalization by @Lazarus-931 in #2277
• Remove computer use and agents examples by @Blaizzy in #2278
• Account for built-in APC cache layouts without fallback by @Blaizzy in #2269
• Use inherited APC memory profiles with specialized cache overrides by @Blaizzy in #2272
• moe_offload: recognise expert tensors in checkpoints without a leading key prefix by @Lazarus-931 in #2284
• Bound unused cache graphs during single-request decoding by @Blaizzy in #2271
• Clarify APC stored and restored token counters by @Blaizzy in #2270
• Guard _extend_cache against caches that cannot batch by @Lazarus-931 in #2286
• Add port of Meta's Sapiens2 dense prediction model(s) by @eklipse2k8 in #2283
• Fix Qwen3 Omni chunked prefill with visual embeds by @lucasnewman in #2265
• Fix combined image and video inputs for Qwen3-Omni by @lucasnewman in #2287
• Fix empty assistant content for Anthropic tool calls by @JakeATX in #2291
• Fix GLM5-Next quantized head sanitization by @JakeATX in #2290
• Add Bonsai 2 27B model by @lucasnewman in #2293
• Reduce test suite and drive model tests from JSON cases by @Blaizzy in #2276
• lfm2_vl: inherit the text LFM2 language model to enable speculative decoding by @tugot17 in #2280
• Discover local models by default and expose loaded status by @Blaizzy in #2294
• qwen3_vl_moe: feed video pixels through the vision tower by @Lazarus-931 in #2299
• Support current moondream2 checkpoints and moondream3 mlx quants by @Lazarus-931 in #2301
• Add support the the LLaDA-Image model family by @lucasnewman in #2306
• Automatically handle repo variants of supported image models by @lucasnewman in #2303
• feat: SAM 3D Objects for MLX by @eklipse2k8 in #2282
• deepseek_vl_v2: change rope_traditional to False by @Lazarus-931 in #2323
• fix(server): return 400 when a chat/omni model is sent to /v1/audio/* by @Lazarus-931 in #2320
• Add Mistral Large 3 (675B VLM) support by @Lazarus-931 in #2308
• docs: explain the server's speculative draft counters by @ykhrustalev in #2324
• Fix Chat and Responses conversation parity by @Blaizzy in #2260
• Add Qwen-Image-2.1 (text-to-image + edit) support by @Lazarus-931 in #2321
• Bump version to 0.7.2 by @lucasnewman in #2326

New Contributors

• @kiarina made their first contribution in #2258
• @ykhrustalev made their first contribution in #2266
• @tugot17 made their first contribution in #2280

Full Changelog: v0.7.1...v0.7.2