0.6.0

oMLX 0.6.0

Compared with 0.6.0rc1, the final release adds community intelligence benchmark publishing and broader Qwen3.8 and Gemma 4 support while improving reasoning controls, embeddings, model discovery, and serving stability.

oMLX 0.6.0 introduces experimental distributed serving, more responsive concurrent inference, lower long-context memory use, new model families, built-in web search, speech-to-text, and community intelligence benchmark publishing.

Distributed Serving

• Added experimental distributed serving across Macs. Models can use tensor or pipeline parallelism with capability-aware planning, selective shard staging, memory guards, and one-click activation. Qwen3.6-27B reached 28.6 tok/s across two Macs versus 16.1 tok/s on one, while a 225 GB MiniMax-M3 checkpoint loaded across 128 GB and 256 GB Macs. By @ashhart in #2423.

• Added heterogeneous Metal and CUDA model pools. Apple Silicon and NVIDIA workers can contribute to one logical pool with memory-aware layer placement, worker enrollment, ConnectX discovery, and NCCL verification. By @ashhart in #2591.

• Added SSD-backed prompt reuse for distributed ranks. Ranks restore only cache prefixes available across the cluster, with incremental storage that stays linear as context grows. #2620.

Distributed serving is included in the DMG but remains experimental and disabled by default. Enable it from Web Dashboard > Global Settings > Advanced > Distributed Inference, then save and restart oMLX.

Community Intelligence Benchmarks

• Intelligence benchmark results can now be published to omlx.ai. Eligible local runs upload a score summary and compressed per-question answers without uploading question text. Small or external-endpoint runs remain private, and both the web dashboard and macOS app show upload status and link to My Submissions. #2693.

Serving Responsiveness and Long-Context Memory

• Decode throughput during concurrent prefill improved by 1.6x to 43x. Prefill now yields GPU time to active decodes and adapts chunk size to a target stall time, while solo prefill performance remains unchanged. #2633.

• Mixed CacheList prefix storage is now linear instead of quadratic. The affected Inkling workload dropped from 282.7 GB at 84K tokens to 15.3 GB at 94K tokens while preserving byte-identical restores. By @Pietroski in #2550.

• GDN recurrent state now uses bounded SSD sidecars by default. The new policy keeps recurrent state separate from ordinary KV storage, with RHT-INT16 reducing storage by 1.93x versus FP32 while restoring in FP32. By @hojin12312 in #2569 and #2644.

• Memory pressure now reclaims MLX buffers before shrinking the hot cache. This avoids repeatedly discarding reusable prompt state while retaining immediate emergency-pressure enforcement. Reported in #2581, fixed by @PhilipJohnBasile in #2635.

Model Support and Performance

• Expanded Qwen3.8 support. oMLX now loads blockwise FP8, embedded MTP, and the mixed ModelOpt NVFP4 unsloth/Qwen3.8-27B-NVFP4 checkpoint directly. NAX prompt throughput for common group-size-128 checkpoints improved from 513 to 925 tok/s in the reported workload. By @scaryrawr, @LKyaha, and @True2456 in #2653, #2659, and #2657.

• DeepSeek V4 prefill is 23–28% faster across tested context lengths. Optimized attention, cache, indexer, and scheduler paths improve prompt throughput, while native handling of unaligned tails reduced peak memory from 142.255 GiB to 105.059 GiB in a 347K-token test. By @DiscoStew6082 and @jonathan308 in #2559, #2562, #2563, #2568, and #2674.

• Added Lightning MTP support for unified Gemma 4 checkpoints. Embedded assistant configurations are preserved, and depth-8 verification now restores rotating caches correctly after rejected drafts. By @djr747 and @adamw3455 in #2683 and #2690.

• Added Ling 3.0 Flash support. FP8, mixed FP4/FP8, oQ/oQe conversion, batching, and SSD prefix-cache restores are supported. The trained SwiGLU clamp improved measured HumanEval accuracy from 71.3% to 88.4%. By @scaryrawr in #2526, #2534, and #2571.

• Added Meta Muse Glimmer 30B VLM support with DFlash. Text, vision, reasoning, ATEM tool calls, prefix caching, and oQ quantization are supported, with about 1.4x decode speedup on the tested oQ4 pair. #2586, #2587, and #2643.

• Added complete Jina Reranker v3.5 support. Projector loading, sliding-window attention, dual matching, block fusion, and reference-score parity are included. By @damascoemi in #2449 and #2494.

• Sentence-transformers checkpoints now honor declared pooling metadata. CLS, last-token, and mean pooling are selected from the checkpoint instead of silently returning incompatible embeddings. By @cedricamram in #2594.

Chat, APIs, Reasoning, and Tool Calling

• Added built-in web search to admin chat. Models can use web_search and fetch_url through DDGS, DuckDuckGo, Brave, or SearXNG, with source cards and redirect-safe URL fetching. External API clients are unchanged. #2596.

• Added file and realtime speech-to-text. Compatible Whisper, Voxtral, and Qwen3-ASR models support file transcription and live microphone input with correct segmentation, language detection, and CJK streaming. #2603.

• Improved reasoning output and controls. Responses API streams prompt-opened thinking through dedicated reasoning events, while the API, web dashboard, and macOS app now preserve free-form or numeric reasoning-effort values such as xhigh and 0.9. #2623, #2675.

macOS App and Updates

• Reworked the self-updater to make bundle replacement crash-safe. Atomic bundle exchange prevents interrupted updates from leaving an incomplete or damaged app. This addresses the macOS 27 failures reported in #2522.

Affected macOS 27 users upgrading from a build older than 0.6.0rc1 should install 0.6.0 manually from the DMG once. The new updater handles subsequent releases safely.

Miscellaneous

• Improved pressure handling, DeepSeek V4 prefill memory estimates, cache cleanup, hot-cache statistics, repeated aborts, and active-engine unload safety. #2512, #2639.

• Fixed DeepSeek V4 generation anchors, tool-turn round trips, parser edge cases, and thinking-mode benchmark extraction. #2510, #2513, #2661.

• Fixed NVFP4 oQ sensitivity analysis, incomplete sharded-model discovery, default macOS cluster node names, and Homebrew installation guidance. #2433, #2432, #2679, #2694.

• Improved accessibility, Chinese typography, localization, network binding, Dock behavior, and Codex/Claude launcher integration. #2542, #2671, #2480.

• Reorganized Global Settings, refreshed model presets, added MCP exposure and chat tool-round controls, honored web-search result limits, and surfaced unenforced Responses API formats. #2560, #2699, #2696, #2702.

• Improved cluster runtime discovery, mixed Python minor-version support, version status reporting, and macOS server restart recovery. #2697, #2704, #2705.

Upgrade Notes

• Distributed inference remains experimental and disabled by default. Enable it under Global Settings > Advanced, save, and restart oMLX.

• Decode fairness is enabled by default. Set decode_fairness=false to restore the previous concurrent-prefill behavior.

• GDN storage defaults to auto with RHT-INT16. Existing explicit storage and FP32 settings are preserved.

• Legacy mixed-CacheList SSD blocks are invalidated automatically because the new linear format is incompatible.

• DeepSeek V4 temperature-0 output may differ at near-tied token choices because optimized kernels change floating-point reduction order.

• Users running 0.6.0rc1 can upgrade directly. No manual settings migration is otherwise required.

New Contributors

Thank you to @Maxim-Mazurok, @damascoemi, @jetztmalehrlich, @Pietroski, @studioburnside, @tannerdsilva, @LXD-8, @luziyi, @DiscoStew6082, @fxd0h, @mvdbos, @ashhart, @PhilipJohnBasile, @mkmsyk, @nazerim, @leepokai, @dawgflymd, @adamw3455, @djr747, @LKyaha, @cedricamram, @cgs4github, and @hellodk for their first contributions in this release line.

Full Changelog: v0.5.7...v0.6.0