vLLM Metal v0.28.0

vLLM Metal v0.28.0

v0.28.0 aligns the plugin release number with its vLLM v0.28.0 core dependency. The release contains 152 merged pull requests from 28 contributors since vLLM Metal v0.2.0.

Highlights

• No compiler required for release installs: the vllm-metal wheel contains its native extensions and precompiled Metal libraries, and the installer uses a prebuilt macOS arm64 vLLM wheel (#423, #508).
• Distributed serving: Metal gained vLLM Ray-executor support for its validated single-node configuration, pipeline parallelism, and dense data-parallel serving with one replica per Mac (#417, #427, #451).
• Speculative decoding: draft-model and n-gram proposers landed, Gemma 4 MTP gained multi-step drafting, and draft KV became scheduler-managed with bounded cold-ingest paths (#447, #456, #658, #630, #643).
• GGUF serving: MLX-native GGUF loading expanded from local Q8_0/Q4_0 models to Q4_1, Llama/Mistral mappings, and remote GGUF references (#453, #480, #498, #546, #635).
• M5 NAX prefill: an automatic paged-attention prefill kernel uses M5 NAX tensor units for supported MHA/GQA/MQA shapes. The reported serving workloads measured 21 to 41 percent lower mean TTFT and 27 to 33 percent higher total throughput where attention was a material part of the step (#623).
• Hybrid prefix caching: Qwen3.5-style SDPA + GDN models can restore recurrent-state checkpoints from scheduler block IDs when prefix caching is enabled (#584).
• Sampling and prefill projection: the deferred decode pipeline gained native temperature/top-k/top-p sampling for supported batches, while paged prefill projects vocabulary logits only for rows that will be sampled (#649, #590).

Installation

Requirements:

• Apple Silicon with macOS 15 or newer.
• Native arm64 Python 3.12.

Stable channel:

curl -fsSL https://raw.githubusercontent.com/vllm-project/vllm-metal/main/install.sh | bash -s -- --stable

The installer downloads prebuilt vllm-metal and vLLM wheels. It does not invoke a C++ or Metal compiler.

Release Artifacts

The release attaches a native Python 3.12 wheel for macOS 15 arm64:

vllm_metal-0.28.0-cp312-cp312-macosx_15_0_arm64.whl

The compatible upstream core is the prebuilt vLLM v0.28.0 macOS arm64 wheel.

Engine, Cache, and Performance

• The decode path gained one-step-ahead sampling, batched paged-prefill sampling, and batched structured-output mask application (#588, #582, #587).
• GDN prefill and decode reduced state handoff overhead through deferred scatter, removed per-layer materialization barriers, and in-place state-row updates (#620, #632, #634).
• Paged decode fused K/V cache writes and reused KV reads across speculative verification windows (#469, #534).
• Metal memory admission now honors --gpu-memory-utilization and rejects scheduler layouts larger than the allocated cache pool (#514, #527).
• Native non-greedy sampling remains opt-in and retains the torch path for seeded requests, penalties, logprobs, token constraints, and mixed batches (#649).

Models and Serving Features

• LoRA support expanded to QLoRA adapters on AWQ and MLX-quantized bases (#351, #470).
• PaddleOCR-VL gained multimodal and multi-image serving support (#422, #429).
• EXAONE 4.0 and the Laguna model family were added (#459, #562).
• Qwen3.8 hybrid models were documented and validated on Metal (#609).
• BGE-M3 gained dense embedding and sparse token-classification pooling through the encoder backend (#612).

Compatibility and Operational Changes

• vllm-metal and vLLM core are both versioned 0.28.0 for this release.
• MLX is pinned to exactly 0.32.0, matching the prebuilt native extension ABI.
• The prebuilt wheel deployment target is macOS 15.
• The optional Rust frontend was removed (#490).
• Source builds remain available through VLLM_METAL_BUILD_FROM_SOURCE; release installs use packaged native artifacts (#540).
• Development and stable releases now use separate channels. Stable releases are draft-first and tied to releases/vX.Y.Z branches (#652).

Known Boundaries

• The distributed modes above retain the topology and feature restrictions documented in their linked PRs; they do not imply unrestricted combinations of DP, PP, TP, LoRA, multimodal, and speculative decoding.
• Hunyuan Q/K normalization attribute names are recognized, but the linked change documents a remaining RoPE/norm ordering mismatch and is not a claim of complete Hunyuan support (#659).

Contributors

Thanks to @Astrowebdeving, @CAICAIIs, @DONGRYEOLLEE1, @KrxGu, @LxYuan0420, @PinJhih, @StevenWang-CY, @WindChimeRan, @abcgco, @adolphinvx, @eplatero97, @ericcurtin, @fdaluiso, @feiiiiii5, @harivilasp, @hyunnnchoi, @mhdimo, @narain1, @renyuchen1, @ricky-chaoju, @ritunjaym, @robertomeroni, @samithaj, @scyyh11, @trevorprater, @tzachicohen, @varad-more, and @zupengwang.

Full changelog: v0.2.0...v0.28.0