v0.4.1

vllm-mlx 0.4.1 collects the latest scheduler, streaming, MTP, parser, memory, and model compatibility improvements merged since 0.4.0.

Highlights

• Preserve active streams and correct finish reasons across SimpleEngine streaming paths.
• Support native mlx-lm chunked prefill while keeping MLLM media generation on its owner thread.
• Improve MTP compatibility for Qwen models, sampled concurrent MLLM decoding, and native status counters.
• Add request-local Poolside v1 parsing and complete GPT-OSS harmony tool-call responses.

Improvements

Streaming and scheduling

• Keep active streaming responses alive through request cleanup.
• Preserve model-provided finish reasons and emit stop for natural completion.
• Support the current native mlx-lm chunked-prefill API.
• Keep incremental MLLM media generation on the model owner thread.

MTP and model compatibility

• Support sampled concurrent MLLM decoding and standard Qwen MTP shard prefixes.
• Pass Qwen pre-norm hidden states to MTP checkpoints.
• Expose cumulative native MTP counters without resetting them when sampling changes replace the batch generator.
• Select text-model families by prefix and require mlx-vlm 0.6.5 for corrected Qwen 3.5 weight loading.

Tool calling and parsing

• Add request-local Poolside v1 reasoning and tool parsers.
• Preserve complete GPT-OSS harmony tool calls through final responses.
• Parse the Mistral [ARGS] format used by Ministral 3 and Devstral Small 2.

Memory and API behavior

• Release MLX embedding buffers after each batch.
• Reconcile registry budgets with Metal allocation limits.
• Resolve embedding and rerank truncation from each model's configuration instead of a fixed length.

Install

pip install vllm-mlx==0.4.1

Harmony support remains available as an optional extra.

pip install "vllm-mlx[harmony]==0.4.1"

Changes

• Poolside v1 request-local parsers (#644)
• MLX embedding buffer cleanup (#667)
• Active streaming response preservation (#666)
• Sampled concurrent MLLM MTP decoding (#662)
• mlx-vlm 0.6.5 compatibility floor (#633, #675)
• Prefix-cache guard regression coverage (#655)
• Standard Qwen MTP shard prefixes (#664)
• Mistral [ARGS] tool-call parsing (#631)
• Complete GPT-OSS harmony tool-call responses (#562)
• Registry and Metal memory budget reconciliation (#696)
• Model-aware embedding and rerank truncation (#626)
• Streaming finish-reason preservation (#681, #629)
• Prefix-based text-model family selection (#686)
• Qwen pre-norm MTP hidden states (#660)
• Native mlx-lm chunked prefill support (#648)
• MLLM owner-thread media streaming (#551)
• Native MTP status counters with cumulative replacement-safe state (#656)

Thanks to @Thump604, @SvaRgoS, @mabaeyens, @CBribiescas, @danmackinlay, @brandy975, @janhilgard, and @mvmories for the contributions included in this release.

Full changelog v0.4.0...v0.4.1