0.5.0

This release focuses on Lightning MTP speculative decoding, oMLX custom-kernel acceleration, oQe imatrix-enhanced quantization, and admin QoL improvements. It also fixes M5-series custom-kernel performance regressions with NAX-aware dispatch paths.

Highlights

• Lightning MTP. Depth-k native speculative decoding now covers Qwen3.6-27B, Qwen3.6-35B-A3B, DeepSeek-V4-Flash, and GLM-5.2 with adaptive depth, sharper draft sampling, and verify-shape Metal kernels adapted from MTPLX. On M3 Ultra, greedy 400-token runs improved Qwen3.6-35B-A3B from 89.6 to 140.4 tok/s and Qwen3.6-27B from 35.0 to 55.1 tok/s. (#2113)
• oMLX Custom Kernels. Added DeepSeek V4, Qwen3.5/3.6, and GLM-5.2 custom-kernel paths, plus NAX-aware dispatch to avoid M5-series regressions. On M3 Ultra, custom kernels improved prefill by up to +45% for DeepSeek-V4-Flash, +33% for Qwen3.6-27B, and +99% for GLM-5.2; the custom-kernel path is most effective on M1-M4 Apple Silicon Macs. (#2048, #2100, #1984, 69f902b)
• oQe imatrix enhanced quantization. oQ now has an activation-importance calibration pass, MoE expert coverage tracking, weighted affine quantization, cache reuse, and admin UI controls. In the PR benchmark, oQ4e improved average accuracy over oQ4 on Gemma-4-26B-A4B-it, Qwen3.5-9B, Qwen3.6-35B-A3B, and Qwen3.6-27B while keeping the same disk-size class. (#2057)
• QoL and comparisons. The admin UI can now benchmark external OpenAI-compatible chat endpoints, and favorite models can be starred so they appear first in model lists and chat pickers.

Performance Snapshot

Area	Benchmark result
Lightning MTP	Qwen3.6-27B: 35.0 -> 55.1 tok/s, 1.57x; Qwen3.6-35B-A3B: 89.6 -> 140.4 tok/s, 1.57x; DeepSeek-V4-Flash: 29.5 -> 35.1 tok/s, 1.19x; GLM-5.2: 15.6 -> 17.9 tok/s, 1.15x. (#2113)
DeepSeek V4 kernels	DeepSeek-V4-Flash-oQ8 prefill improved 313.3 -> 455.8 tok/s at 64k context, a +45% gain, with peak memory reduced from 151.25 GiB to 144.75 GiB. (#2048)
Qwen kernels	Qwen3.6-27B-oQ4e prefill improved 262.2 -> 347.6 tok/s at 64k context, a +33% gain; a 16k parity run generated identical token IDs and improved wall time 47.271s -> 42.505s. (#2100)
GLM-5.2 kernels	16k PP improved 128.1 -> 178.9 tok/s, a 1.40x gain. 32k PP improved 87.7 -> 174.4 tok/s, a +99% gain, with 18.53 GB lower peak memory. (#1984)
oQe accuracy	oQ4e averages: Gemma-4-26B-A4B-it 85.27% vs oQ4 83.49%; Qwen3.5-9B 73.09% vs 70.01%; Qwen3.6-35B-A3B 83.88% vs 82.83%; Qwen3.6-27B 85.62% vs 84.99%. (#2057)

Changes Since 0.5.0rc1

• Improved Lightning MTP adaptive depth selection across different models and hardware. (#2150)
• Fixed Qwen3.5/3.6 and GLM-5.2 custom-kernel performance, stream handling, and compatibility issues. (#2155, #2160, #2162, #2165, #2170)
• Fixed SpecPrefill SSD cache isolation. Contributed by @aosama in #2154.
• Fixed MXFP8 U8 scale decoding during loading and oQ conversion. Contributed by @builtbycnob in #2159.
• Fixed reranker-native chat templates and tool-call cache probing. Contributed by @shyuan in #2147 and @thornad in #2148.
• Fixed the Homebrew spaCy model checksum. (#2151)

New Features

• Added Lightning MTP depth-k native speculative decoding for Qwen3.6, DeepSeek-V4-Flash, and GLM-5.2. (#2113)
• Added bundled DeepSeek V4 MXFP4 MoE and sparse-attention custom kernels. (#2048)
• Added Qwen3.5/3.6 native prefill kernels for FA-256 attention, quantized qmm, GatedDeltaNet, and MoE weighted-sum paths. (#2100)
• Added GLM-5.2 32-head sparse-MLA custom-kernel support for tensor-sharded multi-device runs. Contributed by @aidiffuser in #2070.
• Added NAX-aware dispatch for M5-series Macs, including Qwen3.5/3.6 affine qmm NAX tile variants, FA-256 Steel bypass on NAX GPUs, and DeepSeek V4 MoE routing back to stock NAX-backed MLX kernels for prefill-sized GEMMs. (69f902b)
• Added oQe imatrix-enhanced quantization, calibration cache handling, MoE expert accumulation, and admin controls. (#2057)
• Added external OpenAI-compatible endpoint support to the admin throughput and accuracy/intelligence benchmark flows.
• Added model favorites with star toggles in the web admin and macOS app, including favorites-first sorting in chat/model pickers.
• Added OpenAI-compatible stream=true SSE support to /v1/audio/transcriptions. Contributed by @ethannortharc in #2075, requested by @JohnGalt1717 in #1066.
• Added OpenAI prompt field support for STT vocabulary/context biasing. Contributed by @ethannortharc in #2082.
• Added GET /v1/audio/voices for static TTS voice discovery. Contributed by @ethannortharc in #2084.
• Added Tencent Hy3 / HunYuan V3 support. Contributed by @gilby in #2108.
• Added Ornith model support for oQ conversion and inference. Contributed by @kreeger in #2044.
• Added admin model search, filtering, sortable model tables, persistent sort state, and model visibility controls. Contributed by @jasonpaulso in #2107.
• Added Russian localization for the macOS app. Contributed by @DrMaks22 in #2028.
• Added custom-kernel install documentation and a Homebrew custom-kernel option.

Bug Fixes

• Fixed TurboQuant KV cache + native MTP compatibility, including the VLM verify path that sliced packed TurboQuant states incorrectly. Reported by @kuanjames in #2139.
• Fixed the web admin restriction that prevented enabling TurboQuant KV cache together with native MTP. Reported by @kuanjames in #2133.
• Fixed paged SSD cache signatures so TurboQuant blocks are keyed by bit depth and stale mixed-width cache entries are invalidated. Contributed by @JimStenstrom in #2071, reported by @aosama in #2045.
• Fixed oQe wheel packaging so oqe_calibration_data.json ships with clean installs and missing data fails loudly instead of silently using the smaller corpus. Contributed by @builtbycnob in #2140.
• Fixed malformed parser-emitted tool calls so bad arguments are dropped instead of aborting streaming responses. Contributed by @e345ee in #2142, reported by @daveattard-cell in #2012.
• Fixed Harmony tool-call extraction to require the commentary channel before treating functions.* messages as tool calls. Contributed by @e345ee in #2143, reported by @daveattard-cell in #2032.
• Fixed macOS Models screen load/unload button wrapping for long model names. Contributed by @alan13367 in #2125.
• Fixed DFlash draft checkpoint discovery from the Hugging Face cache. Contributed by @JimStenstrom in #2134.
• Fixed deterministic accuracy-benchmark sampling, with model_settings as an explicit opt-in profile, and kept chat_template_kwargs forwarding intact. Contributed by @JimStenstrom in #1722.
• Fixed forwarding of extra CLI args to Hermes, opencode, Pi, Copilot, and OpenClaw launchers. Requested by @carlyd95 in #2081.
• Fixed request media hardening by restricting API image inputs to inline data URIs. Reported by @anvarknian and @BrianSmithPhotos in #1440.
• Fixed custom-kernel ABI handling for MLX 0.32.0 / nanobind 2.13.0, including ABI mismatch disablement and compiled extension name checks. (#2138)
• Fixed stuck admin/macOS generating status after failed queued requests by clearing prefill tracker rows and republishing snapshots. Reported by @funnygeeker in #2126.
• Fixed decode-shaped multi-row SDPA routing, DeepSeek V4 dotted HC weight remapping, helper model schema hardening, and Qwen35 custom-kernel build checks. (#2130, #2134)
• Fixed Homebrew spaCy/Kokoro packaging, macOS 27 beta install behavior, and Python/CI dependency caps. Contributed by @heyparth1 in #2111, @ethannortharc in #2085, and @mxl in #2122.

New Contributors

Thank you to @GeorgeTheo99, @heyparth1, @mxl, @tbro0815, @aidiffuser, @alan13367, @builtbycnob, @e345ee, and @aosama for their first contributions to oMLX in this release.

Full Changelog: v0.4.5.dev1...v0.5.0