v0.6.0 · Blaizzy mlx-vlm · Discussion #1258 · GitHub

Blaizzy
Jun 1, 2026
Maintainer

What's Changed

fix(server): remove duplicate “server” response header by @goniz in fix(server): remove duplicate “server” response header #1140
Fix top_p_sampling crash on 3D logits in speculative decoding by @ivanfioravanti in Fix top_p_sampling crash on 3D logits in speculative decoding #1141
Use local rotating-cache offsets for Gemma assistant MTP masks by @Thump604 in Use local rotating-cache offsets for Gemma assistant MTP masks #1139
Fix stale Qwen MRoPE rope_deltas in decode by @neilmehta24 in Fix stale Qwen MRoPE rope_deltas in decode #1143
Fix server chat completions text content array handling by @spicyneuron in Fix server chat completions text content array handling #1137
Add server metrics and improve streamed response handling by @Blaizzy in Add server metrics and improve streamed response handling #1145
Fix streamed byte-fallback token handling in server batching by @Blaizzy in Fix streamed byte-fallback token handling in server batching #1147
Fix APC disk cache directory recovery by @Blaizzy in Fix APC disk cache directory recovery #1149
minicpmo / fastvlm: fix pixel cast on quantized language models by @contrapuntal in minicpmo / fastvlm: fix pixel cast on quantized language models #1098
Fix Qwen3 VL vision config deserialization (patch_size ignored) by @lyonsno in Fix Qwen3 VL vision config deserialization (patch_size ignored) #1157
Fix max kv size enforcement by @lucasnewman in Fix max kv size enforcement #1160
[codex] Enforce server context budget by @eloe in [codex] Enforce server context budget #1153
Fix LFM2-VL model by @lucasnewman in Fix LFM2-VL model #1162
Add ZAYA1-VL model support by @Blaizzy in Add ZAYA1-VL model support #1159
Fix an error when loading the MolmoPoint-8B model by @lucasnewman in Fix an error when loading the MolmoPoint-8B model #1164
Fix image feature fp16 overflow in Molmo2 by @lucasnewman in Fix image feature fp16 overflow in Molmo2 #1165
Improve Gemma4 MTP server batching by @Blaizzy in Improve Gemma4 MTP server batching #1166
Add Qwen MTP speculative decoding by @Blaizzy in Add Qwen MTP speculative decoding #1167
Add MiniCPM-V 4.6 support by @pzc163 in Add MiniCPM-V 4.6 support #1058
Refactor speculative decoding utilities by @Blaizzy in Refactor speculative decoding utilities #1169
Fix crash with FastAPI workers when using Qwen3.5 family models by @lucasnewman in Fix crash with FastAPI workers when using Qwen3.5 family models #1168
Add Gemma 4 DFlash + adaptive block sizing by @Blaizzy in Add Gemma 4 DFlash + adaptive block sizing #1176
fix: replace deprecated use_fast=True with backend=torchvision in load_processor by @Jonathangadeaharder in fix: replace deprecated use_fast=True with backend=torchvision in load_processor #1170
Fix weight remapping for the Qwen3.5/3.6 vision tower by @lucasnewman in Fix weight remapping for the Qwen3.5/3.6 vision tower #1179
Fix Nemotron processor metadata loading by @Blaizzy in Fix Nemotron processor metadata loading #1177
Add Gemma 4 EAGLE3 speculative decoding support by @Blaizzy in Add Gemma 4 EAGLE3 speculative decoding support #1180
Split speculative decoding utilities by @Blaizzy in Split speculative decoding utilities #1182
Add Laguna support by @Blaizzy in Add Laguna support #1183
Compatibility bridge for non-VL models by @lucasnewman in Compatibility bridge for non-VL models #1181
Unify LoRA loading for VLM/LM models by @lucasnewman in Unify LoRA loading for VLM/LM models #1186
[mlx_vlm] Expose 'strict' parameter in load() function by @zyguy in [mlx_vlm] Expose 'strict' parameter in load() function #1198
Add server support for Anthropic-style /v1/messages API by @lucasnewman in Add server support for Anthropic-style /v1/messages API #1196
Support /v1/responses stateful responses API in the server by @lucasnewman in Support /v1/responses stateful responses API in the server #1199
Fix /v1/messages tool call image handling by @lucasnewman in Fix /v1/messages tool call image handling #1200
Fix Qwen MTP verification drift and sampling parity by @Blaizzy in Fix Qwen MTP verification drift and sampling parity #1188
Modularize server by @lucasnewman in Modularize server #1203
Fix mask creation for batched quantized kv caches by @lucasnewman in Fix mask creation for batched quantized kv caches #1208
Track APC cached tokens for generation metrics by @spicyneuron in Track APC cached tokens for generation metrics #1209
fastvlm: support untied lm_head for the 7B variant by @contrapuntal in fastvlm: support untied lm_head for the 7B variant #1193
Add RT-DETRv2 detection model by @leonnoirclerc in Add RT-DETRv2 detection model #1195
Handle base64-encoded input audio in chat completions by @lucasnewman in Handle base64-encoded input audio in chat completions #1211
Fix inverted --verbose flag on chat / generate / video_generate CLIs by @SuperMarioYL in Fix inverted --verbose flag on chat / generate / video_generate CLIs #1152
Fix for Gemma4 audio task by @lifeiteng in Fix for Gemma4 audio task #980
Add MiniCPM-V 4.6 language wrapper by @Blaizzy in Add MiniCPM-V 4.6 language wrapper #1212
Support Qwen image pixel overrides by @Blaizzy in Support Qwen image pixel overrides #1213
Improve GLM-OCR generation and repetition controls by @Blaizzy in Improve GLM-OCR generation and repetition controls #1214
Fix LFM2-VL image preprocessing for variants shipping input_data_format=channels_last by @contrapuntal in Fix LFM2-VL image preprocessing for variants shipping input_data_format=channels_last #1190
Fix rotation-basis mismatch in TurboQuant L=1 fast quantize by @nnorris7 in Fix rotation-basis mismatch in TurboQuant L=1 fast quantize #973
fix: fallback to defaults by @GeneCodeSavvy in fix: fallback to defaults #1221
Refactor MRoPE handling by @Blaizzy in Refactor MRoPE handling #1135
Fix server finish_reason inconsistencies by @spicyneuron in Fix server finish_reason inconsistencies #1215
Fix inference cancellation on stream abort by @spicyneuron in Fix inference cancellation on stream abort #1217
Add DeepSeek V4 language model by @Blaizzy in Add DeepSeek V4 language model #1223
Add spec-compliant usage and timings for server endpoints by @spicyneuron in Add spec-compliant usage and timings for server endpoints #1216
Add DeepSeek V4 MTP support by @Blaizzy in Add DeepSeek V4 MTP support #1225
Freeze MaskedEmbedder.token_ordering so optimizer can't corrupt it by @DirectriX01 in Freeze MaskedEmbedder.token_ordering so optimizer can't corrupt it #1194
Validate MTP drafter compatibility by @Blaizzy in Validate MTP drafter compatibility #1227
Restructure generation into AR and diffusion engines by @Blaizzy in Restructure generation into AR and diffusion engines #1229
Add support for image generation with PrismML Bonsai by @lucasnewman in Add support for image generation with PrismML Bonsai #1226
Enforce thinking budget in server batching by @Blaizzy in Enforce thinking budget in server batching #1228
Add LLaDA2.X support by @Blaizzy in Add LLaDA2.X support #1230
Add support for FLUX.2 base & klein models by @lucasnewman in Add support for FLUX.2 base & klein models #1232
Improve image generation quality for Bonsai by @lucasnewman in Improve image generation quality for Bonsai #1234
Add support for image editing with FLUX.2 models by @lucasnewman in Add support for image editing with FLUX.2 models #1236
Improve masked diffusion text generation controls by @Blaizzy in Improve masked diffusion text generation controls #1235
Add HRM-Text model support by @Blaizzy in Add HRM-Text model support #1238
Add LFM2 MoE language model by @Blaizzy in Add LFM2 MoE language model #1237
Add Step-3.7 Flash support by @ivanfioravanti in Add Step-3.7 Flash support #1245
fix(kernels): match PyTorch bicubic coefficient (-0.75 non-AA / -0.5 AA) — closes bicubic_interpolate uses a=-0.5 (Keys') instead of PyTorch's a=-0.75 — degrades MoonViT pos-emb parity #1241 by @beshkenadze in fix(kernels): match PyTorch bicubic coefficient (-0.75 non-AA / -0.5 AA) — closes #1241 #1243
Fix(qwen3_omni_moe): enable audio input (mask key + audio placeholder) by @yuhuanowo in Fix(qwen3_omni_moe): enable audio input (mask key + audio placeholder) #1240
docs: align Python version metadata by @aqilaziz in docs: align Python version metadata #1150
Fix ByteLevel streaming detokenizer detection for Step 3.7 Flash by @ivanfioravanti in Fix ByteLevel streaming detokenizer detection for Step 3.7 Flash #1246
turboquant: guard L=1 value kernels behind not use_rht (fix masked decode under RHT) by @popfido in turboquant: guard L=1 value kernels behind not use_rht (fix masked decode under RHT) #1244
Fix cache merging for batched quantized caches by @lucasnewman in Fix cache merging for batched quantized caches #1248
security: require patched Starlette by @Thump604 in security: require patched Starlette #1249
Don't show tqdm progress bars for prefill by default by @lucasnewman in Don't show tqdm progress bars for prefill by default #1250
Fix adaptors with the /v1/responses API by @lucasnewman in Fix adaptors with the /v1/responses API #1251
[codex] Fix models endpoint for loaded local models by @Blaizzy in [codex] Fix models endpoint for loaded local models #1253
perf(turboquant): RHT-correct L=1 value kernels (keep the fast path under RHT) by @popfido in perf(turboquant): RHT-correct L=1 value kernels (keep the fast path under RHT) #1252
Fix trainer adapter save fallback by @Blaizzy in Fix trainer adapter save fallback #1255
Fix Qwen MTP batched target-verify drift by @Blaizzy in Fix Qwen MTP batched target-verify drift #1210
Add Nemotron Labs Diffusion model by @Blaizzy in Add Nemotron Labs Diffusion model #1239
Update version to v0.6.0 by @Blaizzy in Update version to v0.6.0 #1257

New Contributors

@contrapuntal made their first contribution in minicpmo / fastvlm: fix pixel cast on quantized language models #1098
@lyonsno made their first contribution in Fix Qwen3 VL vision config deserialization (patch_size ignored) #1157
@pzc163 made their first contribution in Add MiniCPM-V 4.6 support #1058
@Jonathangadeaharder made their first contribution in fix: replace deprecated use_fast=True with backend=torchvision in load_processor #1170
@zyguy made their first contribution in [mlx_vlm] Expose 'strict' parameter in load() function #1198
@leonnoirclerc made their first contribution in Add RT-DETRv2 detection model #1195
@SuperMarioYL made their first contribution in Fix inverted --verbose flag on chat / generate / video_generate CLIs #1152
@lifeiteng made their first contribution in Fix for Gemma4 audio task #980
@nnorris7 made their first contribution in Fix rotation-basis mismatch in TurboQuant L=1 fast quantize #973
@GeneCodeSavvy made their first contribution in fix: fallback to defaults #1221
@DirectriX01 made their first contribution in Freeze MaskedEmbedder.token_ordering so optimizer can't corrupt it #1194
@beshkenadze made their first contribution in fix(kernels): match PyTorch bicubic coefficient (-0.75 non-AA / -0.5 AA) — closes #1241 #1243
@yuhuanowo made their first contribution in Fix(qwen3_omni_moe): enable audio input (mask key + audio placeholder) #1240
@aqilaziz made their first contribution in docs: align Python version metadata #1150
@popfido made their first contribution in turboquant: guard L=1 value kernels behind not use_rht (fix masked decode under RHT) #1244

Full Changelog: v0.5.0...v0.6.0

This discussion was created from the release v0.6.0.

Replies: 1 comment

nikolaydubina
Jun 4, 2026

thank you for doing this 👍🏻

0 replies

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment