Repository navigation
v1.0.3
What's new in v1.0.3
Rapid-MLX 1.0.3 adds HunyuanVideo 1.5 text-to-video, faster decode on qualified models, and more reliable tool calling and structured answers. Desktop can share its API with devices on your network, answer questions about a selected indexed folder, and show where memory is going. Downloads, speech, model selection and first-run CLI chat also get fixes.
Download: Rapid-MLX Desktop for macOS | CLI: rapid-mlx upgrade or curl -fsSL https://rapidmlx.com/install.sh | bash
Video: HunyuanVideo 1.5
HunyuanVideo 1.5 generates 480p video from text on Apple Silicon Macs with 64 GB or more unified memory. Both hunyuan-video-1.5-480p and hunyuan-video-1.5-480p-distilled use the existing video jobs, progress, cancellation and MP4 output. The base checkpoint downloads about 53 GB directly from Hugging Face (#5679, #5687).
The CFG-distilled variant runs one transformer pass per step instead of two; it still uses 50 steps. On an M3 Ultra with 512 GB, at 848×480, 33 frames, 50 steps and seed 42, the measured times were (#5687):
| Variant | Load | Denoise | Video decode | Peak memory |
|---|---|---|---|---|
| Base | 16 s | 1,331 s | 61 s | 20.2 GB |
| CFG-distilled | 9 s | 657 s | 63 s | 20.2 GB |
This release supports text-to-video only for this family. The Tencent Hunyuan Community License excludes the European Union, United Kingdom and South Korea (#5679).
Faster Decode on Qualified Models
Dense Qwen3.5 models gain guarded decode fusions when speculative decoding is off. Qwen3.5/3.6 35B-A3B MoE models gain measured, device-specific tuning for plain decode. These are separate measurements, each against its own baseline, with a 564-token prompt, 256 greedy output tokens and a fresh server per repetition (#5690, #5728).
| Model / change | Chip | Before tok/s | After tok/s | Change |
|---|---|---|---|---|
| Qwen3.5 4B 4-bit, dense fusions | M3 Ultra | 166.89 | 187.86 | +12.6% |
| Qwen3.5 4B 4-bit, dense fusions | M4 Pro | 81.76 | 85.68 | +4.8% |
| 35B-A3B 4-bit, singleton tuning | M3 Ultra | 157.19 | 177.00 | +12.6% |
| 35B-A3B 4-bit, singleton tuning | M5 Max | 167.27 | 171.81 | +2.71% |
The singleton tuning left MTP throughput within spread on M3 Ultra and essentially unchanged on M5 Max; its small M4 Pro gain stayed below the enablement threshold. These gains are not additive (#5728).
Compatible dense Qwen3.5, LFM2.5, LFM3 and Gemma 4 decode also combines residual normalization and supported projections, with smaller measured gains on some chips. MTP avoids unused draft-head work while parked, uses continuously measured verification costs, and enables guarded fused 4-bit MoE experts by default. The expert path uses approximate arithmetic, so output can differ slightly from the unfused path (#5695, #5674, #5689, #5694).
More Reliable Agent Tool Calls
Tool calls that use dotted or slash-separated names, as some agent prompts write them, now resolve to the single matching declared tool after separators are normalized to underscores. Exact names take precedence; ambiguous and unknown names keep the existing behavior (#5788).
Gemma 3 12B and 27B gain structured tool calls, including streamed calls, parallel calls and tool-result history. The smaller 1B and 4B catalog variants stop advertising tool support they could not reliably produce. The new parser keeps quoted brackets inside arguments and fenced examples in prose consistent between streaming and non-streaming (#5715, #5750).
Forced and required tool choices on the Anthropic-compatible API now generate arguments instead of trying to invent an empty call. Automatic tool choice also works when parallel calls are disabled. Existing Qwen and GLM tool parsers pass undeclared tool names to the client so it can return a tool error and let the model recover; the built-in agent still refuses to run unknown tools (#5724, #5723, #5700).
Desktop: LAN Access, Folder Chat and Memory
LAN access is an opt-in setting under Settings → Tools. It restarts the server with a persistent bearer key saved in Keychain and shows addresses and copyable connection details for other devices on your network. Network clients can use inference, while model management and other administrative routes stay on this Mac. Network access is off by default and uses plain HTTP (#5676). Local search, including grounded chat, stays available only on this Mac (#5749).
Ask this folder… starts a chat scoped to one registered Local Search folder, with a removable folder chip and saved passage previews below each answer. It requires Built-in Agent Harness (Experimental) in Settings. Folder questions are text-only with tools disabled; no matching evidence produces an abstention without running inference. Saved excerpts survive changes to the original file or index, so you can inspect the evidence used for that answer; passages do not guarantee that the model's answer is correct (#5654).
API clients can also request grounded chat from explicitly selected registered folders, with citation IDs and source locators in JSON and streaming responses. Local Search can return full matched chunks on request, without reopening the original files (#5652, #5638).
Performance settings now combines the Mac's app, wired, compressed and unused memory with engine Metal allocations, prompt-prefix cache usage, running and waiting requests, and a five-minute trend. Engine allocations overlap the Mac totals; they are not extra memory to add to them. Missing measurements show a dash, and the panel supports VoiceOver and Simplified Chinese (#5651, #5677).
New Features and Improvements
- Open an existing checkpoint: Settings → Models → Open existing checkpoint… registers a complete local MLX chat checkpoint without copying weights or changing the managed download folder. Remove reference leaves its files intact; missing or incomplete checkpoints report a problem instead of starting a download (#5660).
- Measured speed references: Desktop recommendations and CLI
modelsanddoctorshow offline references matched to chip, RAM and model configuration. After chat, Generation details saves supported execution facts and timing with the answer. References come from community measurements refreshed on November 11, 2026. Sparse or unmatched data stays unavailable; the ranges are references, not performance guarantees (#5647, #5655, #5659, #5782). - Community Benchmark comparisons:
rapid-mlx benchmark run --configs N(2–16) measures your current Rapid Lattice configuration against nearby configurations on the same machine, with interleaved fresh-load repetitions (--repetitions, 3–10), so submissions can show whether a different setting is faster. The default stays a quick single-configuration run (#5763). - Benchmark tab in Desktop: Benchmark is now a regular sidebar tab next to Agent instead of an experimental setting. Measure models on your Mac and, if you choose, share results with the Community Benchmark; nothing runs until you start (#5781).
- Faster default on base and Pro Macs: with 48 GB or more on a base or Pro chip, recommendations now lead with Qwen3.6 35B-A3B (35B total, 3B active), which community measurements show at roughly 59–104 tok/s versus 8–17 tok/s for the dense 27B on the same chips. Max and Ultra Macs keep the dense 27B first. MoE models now show total and active parameters in Desktop and the CLI (#5784).
- Idle speech release: transcription, alignment and speech-synthesis models can unload while the API stays online, then reload on the next request. Desktop uses this with its idle setting on supported runtimes; CLI users opt in with
--audio-lane-idle-ttl(#5635). - Consistent CLI first run: the installer and cold-cache chat now use the LFM2.5 2.6B starter, about a 1.6 GB download. Installer recommendations use the smaller starter below 8 GB or when RAM is unknown; existing downloaded chat models can still be reused (#5631, #5632).
- Download checks and recovery:
pull --checkincludes free disk space, corrupt cached files get one targeted repair attempt, and Desktop automatically falls back from an unforced failing mirror to Hugging Face within its retry budget (#5725, #5736, #5666). - Speech compatibility: standard TTS names
tts-1,tts-1-hdandgpt-4o-mini-ttsresolve to the default local speech model. Audio-enabled installer runs prepare Kokoro's English prerequisites; missing prerequisites get actionable repair instructions (#5712, #5713). - Privacy: consented anonymous metrics now include Desktop launches and daily foreground use, plus coarse startup and agent-readiness failure categories. Diagnostic categories exclude raw messages, paths and credentials; concurrent first runs share one install identity (#5697, #5634, #5698, #5668, #5709).
Bug Fixes
- Ask this folder: small folders now decline unrelated questions instead of answering with citations to irrelevant files (#5767).
- Desktop recognizes models in a custom models folder after unloading and lets the next chat start them again (#5768).
- JSON output requests now use the same thinking preference as plain chat, avoiding unexpected reasoning that can leave the answer empty.
pullandchatnow accept the suggested--force-disk-checkoption to proceed past insufficient-disk warnings.- Structured output preserves reasoning and the user's system-prompt prefix, including on vision models. JSON constraints start after reasoning ends and accept the separator before the answer; fallback guidance is reported in a warning header on multimodal requests (#5675, #5678, #5682, #5707).
- Strict Responses streams no longer duplicate JSON or append reasoning to it. Strict schema violations return HTTP 422
json_schema_violationinstead of a retryable gateway error (#5706, #5727). - Vision requests enforce the reasoning budget during generation, leaving room for the answer. Casual-chat thinking decisions follow the requested model on multi-model servers, and CLI chat hides reasoning when thinking is off. Models that always reason get a larger default CLI token budget (#5730, #5739, #5716, #5747).
- Anthropic-compatible sessions resume with
redacted_thinkingblocks in assistant history (#5743). - Qwen XML tool parsing rejects ambiguous parameters instead of rewriting arguments, and accepts exactly integral decimal values for integer parameters (#5636, #5699).
- Agent setup skips servers that need a missing key or serve non-chat models and gives a working next step. Process discovery recognizes actual Rapid-MLX servers and preserves repository identities when matching models (#5594, #5605).
- Desktop model search results appear directly below the search box. Long repository names keep their distinguishing text, raw repositories retain their identity, and colliding menu titles get distinct labels (#5633, #5669, #5733).
- Pending or denied external-volume access shows an explanation and Retry instead of hanging model discovery or treating unreadable models as missing (#5670).
- Desktop downloads and onboarding now explain denied or pending cache-volume access before pulling and after a stall, with permission remedies and a fresh access check on Retry (#5765).
- Deleting a Desktop server-backed conversation also requests server deletion and revokes saved-passage access once confirmed. Offline or unconfirmed deletion stays pending with Retry across relaunch; original files and the search index remain intact, and retained database and attachment bytes are not physically purged (#5658).
- Mirror downloads report disk-full errors clearly and retain partial files for retry. Removing a subfolder model variant preserves other cached variants and shared files (#5746, #5735).
- Dynamic model loads use a default memory budget, evicting idle unpinned models or returning HTTP 507 when admission fails instead of risking an overcommit (#5701).
- Whisper large-v3 turbo rejects translation requests with a clear remedy: use Whisper large-v3 for translation or the transcription endpoint for source-language text (#5719).
- Media errors explain the allowed local-media root and direct audio users to transcribe before chatting (#5740).
- CJK emphasis renders correctly next to symbols, with less repeated work on long streamed messages (#5708).
- Qwen4 MTP restores both attention and index caches after rejected drafts (#5646).
doctoraccepts valid editable contributor installs, and successful routine health probes no longer flood INFO access logs; failed probes remain visible (#5673, #5648).
Upgrade Notes
- Catalog names: 44 niche or superseded aliases leave discovery; their repository IDs retain the same execution profiles. Old names warn and forward for one release (#5684).
- GLM 4.7 Flash:
glm4.7-flash-4bitreplaces the misleadingglm4.7-9b-4bitname and is correctly marked as MoE. It is the same 31.2B-parameter checkpoint, about 16.9 GB at 4-bit; existing downloads are reused and the old name forwards for one release (#5748). - Always-reasoning chat models: when no explicit token limit is given, CLI chat raises its default from 2,048 to 4,096 tokens if the served model cannot turn thinking off. An explicit
--max-tokensstill wins (#5747). - Model residency: without an explicit limit, dynamic loading uses the smaller of 80% of physical RAM and 90% of Metal's recommended working set. Explicit memory limits retain their existing behavior (#5701).
All changes
- chore: bump version to 1.0.3 (#5772) (2420894)
- refactor(lattice): move engine memory policy constants into universal cells (#5785) (d35abd6)
- feat(recommendations): prefer the MoE pick on base/Pro Macs and label MoE parameters (#5784) (ef079da)
- fix(tools): map dotted tool names to a unique declared tool (#5786) (#5788) (2b93697)
- refactor(lattice): move dense/MoE fusion enablement into lattice knobs (#5780) (09d61a6)
- feat(desktop): make Benchmark a regular sidebar tab (#5781) (379039b)
- chore(data): refresh offline decode speed references (2026-11-11) (#5782) (181c5b0)
- refactor(lattice): move plain GDN enrollment into lattice knobs (#5779) (f3eec1f)
- fix(bench): verify model sources of paired comparison runs before upload (#5778) (7112ed5)
- refactor(lattice): move attention KV/query crossovers into lattice knobs (#5777) (a3fdbae)
- feat(lattice): Qwen3.5 eager layer dispatch reads its policy from the lattice (#5776) (eef2ff3)
- feat(lattice): universal hardware, partial geometry and row-range selectors (#5775) (a56e969)
- test(lattice): pin current dispatch at every hard-coded gate boundary (#5774) (e77d908)
- refactor(lattice): unify the adaptive configuration framework as Rapid Lattice (#5773) (7b25e1b)
- feat(bench): compare the current tuning configuration with nearby cells (#5763) (ea9c0b6)
- perf(mtp): deterministic MTP depth and faster MTP from measured tuning cells (#5771) (3fa864e)
- fix(mac): recognize custom-folder models after unload and allow restart (#5770) (27067c8)
- fix(local-search): abstain on unrelated questions in small folders (#5767) (#5769) (e2b861d)
- fix(mac): explain unreadable cache volumes instead of blaming the network (#5765) (#5766) (322273e)
- fix(bench): refuse to share benchmarks of private or gated model repos (#5762) (3354933)
- fix(server): let LAN clients reach model lookup and video status (#5761) (#5764) (a445b5c)
- feat(tuning): explicit cells resolved from a versioned profile (#5760) (e299a09)
- fix(bench): preserve accepted model identity facts when publishing (#5759) (1a2f288)
- fix(bench): record quantization and revision from the loaded checkpoint (#5757) (b57f2ea)
- perf(mtp): enable qualified Flash-Next K1 by default (#5758) (b2ddb2b)
- perf(tuning): measured MTP max-depth knob for the 35B MoE (#5742) (1cbe833)
- fix(cli): accept --force-disk-check on pull and chat (#5754) (35f4c0f)
- fix(tools): make pythonic streaming agree with non-streaming parsing (#5750) (68f72cd)
- fix(chat): keep casual-chat thinking default for JSON output requests (#5756) (280fdb0)
- docs(release-notes): draft 1.0.3 notes (#5751) (aa78309)
- fix(catalog): rename glm4.7-9b-4bit to glm4.7-flash-4bit and mark it MoE (#5748) (0a72db5)
- fix(lan): keep local-search grounding on this Mac (#5749) (2cecfc5)
- fix(models): report out-of-disk during a mirror-only download (#5746) (1561f63)
- fix(chat): raise the default budget for models that always reason (#5747) (fae5fcd)
- fix(anthropic): accept redacted_thinking blocks in assistant history (#5743) (39e1929)
- fix(anthropic): force pinned and any tool_choice through generation (#5724) (fd41aa1)
- fix(rm): remove only the selected variant of a multi-variant repo (#5735) (d2db675)
- docs(media): document local-path and chat-audio limits; make errors actionable (#5740) (9638a00)
- fix(thinking): gate casual-chat auto-disable on the targeted model (#5739) (66172be)
- fix(mac): keep picker titles distinct for raw repos and collisions (#5733) (0fd5743)
- fix(pull): repair a corrupt cached file once instead of failing opaquely (#5736) (26b3315)
- perf: measured tuning layer + singleton MoE backbone speedup (+12.6% plain on M3 Ultra) (#5728) (bb6ea90)
- fix(mllm): apply the decode-time reasoning budget on the vision lane (#5730) (b7d9919)
- fix(pull): include free-disk admission in pull --check (#5725) (1ec3cfb)
- fix(responses): return the shared 422 envelope for strict schema violations (#5727) (ed8e71d)
- fix(tools): keep auto grammar when parallel_tool_calls is false (#5723) (fe42ff6)
- fix(tools): structured tool calls for Gemma 3 12B/27B (#5715) (c01f152)
- fix(tools): forward undeclared tool calls so clients can recover (#5700) (de7110c)
- fix(telemetry): elect one client id when first runs race (#5709) (c67447f)
- fix(audio): reject translation on Whisper large-v3 turbo (#5719) (16e958a)
- fix(cli): hide reasoning in chat when thinking is off (#5711) (#5716) (c5f5803)
- fix(runtime): guard dynamic model loads with a default memory budget (#5701) (6f24f41)
- fix(audio): prepare Kokoro English TTS prerequisites at install time (#5713) (339465c)
- fix(audio): accept OpenAI TTS model names on /v1/audio/speech (#5712) (87e2ea1)
- fix(mac): match cmark punctuation and index sources once in the CJK emphasis pass (#5708) (ec02d80)
- fix(responses): stop strict streaming replay duplicating output text (#5706) (96b9065)
- docs: describe the LFM2.5 2.6B first-chat starter in the README (#5705) (d71050e)
- fix(guided): allow the post-reasoning separator before a JSON schema value (#5707) (bfcad2c)
- chore(models): retire 44 niche/superseded alias names in favor of HF repo ids (#5684) (76d74c9)
- fix(tools): accept integral floats for integer params in qwen3_coder_xml (#5699) (7d221d1)
- perf: fuse residual-add+RMSNorm and same-input projections in dense no-spec decode (#5695) (6ec62b7)
- perf(mtp): continuous measured verify costs; enable fused affine-4 MoE experts by default (#5694) (b21ce06)
- feat(telemetry): locate engine startup failures by boundary and type (#5698) (8fd3f99)
- feat(telemetry): count Desktop launches and daily use in DAU (#5697) (4693a32)
- perf(qwen3.5): enroll guarded GDN fusion for dense no-spec decode (#5690) (721dfbf)
- perf(moe): opt-in fused affine-4 expert decode for Qwen3.6 MoE, incl. MTP verification (#5689) (85373e7)
- feat(video): add CFG-distilled HunyuanVideo 1.5 480p text-to-video (#5687) (5a9fbf5)
- feat(video): add HunyuanVideo 1.5 480p text-to-video on MLX (#5679) (587c9a9)
- fix(guided): gate multimodal schema decoding on /v1/responses when thinking is implicit (#5682) (7b3bdd9)
- fix(mtp): avoid eager head history work while parked (#5674) (890a6e2)
- feat(mac): opt-in, authenticated LAN access for the local API (#5676) (2ed5b02)
- fix(guided): keep reasoning and guidance on multimodal text outputs (#5678) (8455a2d)
- fix(guided): stop rewriting the system prompt for response_format (#5675) (b72596b)
- feat(mac): show a memory overview in Performance settings (#5677) (7eadc4a)
- feat(telemetry): add coarse cause to agent readiness failures (#5668) (efab1d9)
- fix(doctor): accept authenticated editable installs in the import probe (#5673) (8440032)
- fix(mac): don't hang model discovery while volume access is pending (#5670) (3be549e)
- fix(agents): setup skips servers it cannot use and gives a working next step (#5594) (acbdc16)
- fix(mac): keep the distinguishing part of long Hugging Face model names (#5669) (8293099)
- fix(tool-calling): reject ambiguous Qwen XML parameter splicing (#5636) (2952866)
- fix(mac): retry failed mirror downloads from Hugging Face automatically (#5666) (7d2b20c)
- fix(mac): show model search results right under the search box (#5633) (f644ec9)
- fix(cli): start cold-cache chat on the LFM2.5 2.6B starter (#5632) (f97a8a9)
- fix(ci): reject incomplete zero-exit native test runs (#5662) (0d4845c)
- Desktop: open an existing checkpoint without changing downloads (#5660) (71ab9a6)
- Show saved execution-scoped decode references after chat (#5659) (89b517c)
- Expose terminal execution references from actual generation work (#5655) (ad2948d)
- fix(desktop): retain and retry server conversation deletion (#5658) (5037758)
- test(desktop): stop the resident catalog fixture server (#5657) (3febeac)
- Add explicit Desktop folder chat with saved passage previews (#5654) (3264bb3)
- feat(mac): show engine Metal allocator memory in Performance settings (#5651) (1350eed)
- Show chip-specific measured decode references offline (#5647) (3d14a02)
- fix(logging): quiet successful CLI health probes at INFO (#5648) (4272152)
- feat(local-search): ground chat in explicitly selected folders (#5652) (5fcec65)
- docs: audit Homebrew vision dependencies and update guidance (#5649) (a201349)
- fix(mtp): restore Qwen4 head owners after rejected drafts (#5646) (8a98a6b)
- feat(local-search): return full matched text with opt-in locators (#5638) (6baa918)
- feat(audio): unload idle speech lanes while keeping API online (#5635) (aa4d679)
- fix(cli): match only real rapid-mlx serve processes (#5605) (a5181f1)
- fix(telemetry): locate unclassified startup loader failures (#5634) (2a08ce9)
- fix(install): start new installs on the LFM2.5 2.6B starter like Desktop (#5631) (4d5ac49)
Community contributors
Install: brew upgrade rapid-mlx or pip install -U rapid-mlx==1.0.3 (or just rapid-mlx upgrade).
Community: https://discord.gg/nZcXkUjY5R