v0.6.4

What's Changed

• Add support for TTS and STT endpoints in the server by @lucasnewman in #1358
• Add prefill policy support for diffusion models by @lucasnewman in #1357
• Fix diffusion mxfp4 embedding dequantization by @Blaizzy in #1356
• fix: guard cache_offset == 0 against array truth-value error in continuous batching by @imi4u36d in #1354
• Fix DiffusionGemma generation from the server by @lucasnewman in #1368
• Improve diffusion unmasking visualization by @Blaizzy in #1361
• [Cohere] Fix streaming thinking parsing after prompt-opened by @Terrencezzj in #1369
• [Fix] interleaved reasoning by @Terrencezzj in #1370
• Fix color shift from tiled VAE in Flux.2 models by @lucasnewman in #1381
• Revert "[Cohere] Fix streaming thinking parsing after prompt-opened" by @Blaizzy in #1389
• Fix Gemma 4 namespaced tool calls by @Blaizzy in #1379
• Forward strict flag when loading weights by @Blaizzy in #1345
• Add optional API key for the server by @lucasnewman in #1385
• Fix SFT masks for Gemma4 unified by @kqw8 in #1392
• Add preloading for multiple model kinds in the server by @lucasnewman in #1372
• fixes #1404: wire --prefill-step-size into BatchGenerator constructor by @royteeuwen in #1405
• fixes #1402: Every first request after server startup causes a full model reload by @royteeuwen in #1403
• fixes #1406: move draft-kind/draft-block-size env vars outside draft-model guard by @royteeuwen in #1407
• fix(StoppingCriteria): copy eos_token_ids to prevent caller-list aliasing by @raullenchai in #1410
• [performance] Optimize PaddleOCR-VL vision fast paths by @scyyh11 in #1396
• Avoid clearing MLX cache from stream finalizers by @Blaizzy in #1412
• Add Moondream2 model support by @Abdullah-Alharbi61 in #1395
• Qwen3.5 on CUDA: fix Metal-only crashes + chunked gated-delta prefill (8.5x faster prefill) by @Blaizzy in #1423
• Add reasoning_content to chat messages by @Blaizzy in #1411
• improved turbo quant's prefill by @Lazarus-931 in #1433
• adding hisa to dsa based models(currently only deepseek) by @Lazarus-931 in #1434
• fix for #1446 by @Lazarus-931 in #1447
• Fix Qwen3.5 MoE gate-up sanitization by @Blaizzy in #1450
• Fix Gemma 4 cached vision mask overlay by @Blaizzy in #1445
• Fix stale GenerationBatch processor state by @Blaizzy in #1451
• kimi_k25: 3D MoonViT video support (Fixes #1436) by @Lazarus-931 in #1438
• Fix TurboQuant MTP rollback tail zeroing by @Blaizzy in #1432
• convert: operate on the inner model for text-only wrappers (Fixes #1437) by @Lazarus-931 in #1439
• Fix Qwen3.5 TurboQuant batch rollback trim by @Blaizzy in #1453
• Fix quantized Diffusion Gemma sampler speed by @Blaizzy in #1359
• Remove video generate command by @Blaizzy in #1454
• fix(StoppingCriteria): copy eos_token_ids in reset() to prevent caller-list aliasing by @SuperMarioYL in #1430
• Guard empty MRoPE position batches by @Blaizzy in #1455
• Improve Responses API reasoning and tool stream events by @justin-schroeder in #1458
• Select weight files via model.safetensors.index.json when present(fix #1329) by @Lazarus-931 in #1464
• fix(apc): enable APC prefix cache with MTP speculative decoding by @vedang in #1463
• fix(deepseek_v4): clamp shared-expert SwiGLU to swiglu_limit by @popfido in #1469
• Fix Gemma 4 bare namespaced tool calls by @Blaizzy in #1473
• fix(qwen3_5_moe): support split expert weights for ornith 1.0 35b by @scaryrawr in #1472
• Minimax m3 support by @ivanfioravanti in #1374
• Add Unlimited-OCR model support by @shuuul in #1427
• fix(qwen3-omni): eval mRoPE position_ids before the thinker forward by @yuhuanowo in #1461
• fix(rope_utils): make M-RoPE apply differentiable for training by @Lazarus-931 in #1475
• Fix Qwen3-Omni repeated generation by @Blaizzy in #1476
• Fix DiffusionGemma tool marker tokenization by @Blaizzy in #1477
• Constrain transformers below 5.13 by @Blaizzy in #1488
• fix #1480 by @Lazarus-931 in #1484
• Make Qwen MRoPE request-local under continuous batching by @Blaizzy in #1486
• fix #1481: quantized KV cache crashes in mask-length checks by @Lazarus-931 in #1483
• Reject empty OpenAI server inputs by @Blaizzy in #1491
• Consolidate caches + RoPE handling by @lucasnewman in #1494
• fix(lfm2_vl): only create projector LayerNorm when projector_use_laye… by @Lazarus-931 in #1493
• Bump version to 0.6.4 by @Blaizzy in #1499
• Fix MolmoPoint EOS config by @Blaizzy in #1500
• Fix Phi-3-V chat stop tokens by @Blaizzy in #1501
• Always sanitize loaded weights and simplify audio preprocessing by @Blaizzy in #1498
• Add torch-free Pixtral processor by @Blaizzy in #1502
• Support Gemma 4 variable soft-token budgets by @Blaizzy in #1503
• Fix DiffusionGemma processor and video inputs end-to-end by @Blaizzy in #1492
• Fix MiniCPM-V 4.6 weight sanitization by @Blaizzy in #1506
• Use model type in diffusion text-only error by @Blaizzy in #1507
• Unify diffusion generation path by @Blaizzy in #1508
• migration from mlx-lm dependency by @Lazarus-931 in #1509
• Remove obsolete test mocks for safetensors by @lucasnewman in #1511
• Port glm_moe_dsa language model by @Blaizzy in #1390
• Fix computer use action parsing by @Blaizzy in #1514
• Add qwen2 model type and remove mlx-lm from fastvlm by @Lazarus-931 in #1516

New Contributors

• @imi4u36d made their first contribution in #1354
• @kqw8 made their first contribution in #1392
• @royteeuwen made their first contribution in #1405
• @raullenchai made their first contribution in #1410
• @Abdullah-Alharbi61 made their first contribution in #1395
• @Lazarus-931 made their first contribution in #1433
• @justin-schroeder made their first contribution in #1458
• @vedang made their first contribution in #1463
• @scaryrawr made their first contribution in #1472
• @shuuul made their first contribution in #1427

Full Changelog: v0.6.3...v0.6.4