v0.13.4

What's new in v0.13.4

Rapid-MLX 0.13.4

Rapid-MLX 0.13.4 makes qualified local models faster under concurrent work,
adds an experimental end-to-end video workspace, and turns Community
Benchmark into a private-by-default local workspace with explicit sharing.
It also improves memory fitting, model discovery, Markdown rendering, and
headless operation.

Highlights

Faster qualified Qwen models by default

The four exact qualified Qwen3.5/Qwen3.6/Qwen3.8 artifacts now select their
validated MTP preset and continuous scheduler automatically when no
speculative-decoding option is supplied. Across mixed four-request cohorts,
aggregate throughput improved by 14.1%–30.8%, with no ordinary-pass case
becoming a continuous-mode failure. CLI and Server users can restore ordinary
decoding with --no-spec-decode; Desktop exposes the same persistent off
switch.

Qualified artifact	Aggregate throughput change
Qwen3.5 4B 4-bit	+30.8%
Qwen3.5 9B 4-bit	+23.9%
Qwen3.6 27B 4-bit	+14.1%
Qwen3.8 27B 4-bit	+25.9%

For Qwen3.8 27B, the qualified single-request MTP path also scales with
context: measured decode throughput was 1.43× the 0.13.3 ordinary path at
128 tokens of context and 2.34× at 32K. The new continuous scheduler keeps
request/cache ownership transactional across admission, cancellation, and
dynamic joins.

Local video generation in Desktop

Enable Video Generation under Experimental Features to use a dedicated local
workspace with queued jobs, progress, cancellation, restart-safe completed
results, and early memory/workload validation. The signed app includes the
audited runtime and encoder needed for LTX 2.5 q8 (text/image to video),
Wan 2.1 1.3B bf16 (text to video), and CogVideoX-Fun 5B q4 (text to
video), without post-install package provisioning. Wan 2.1 is limited to Macs
with at least 40 GB unified memory.

Community Benchmark, private by default

rapid-mlx benchmark and the new Desktop Community Benchmark page can plan,
run, validate, inspect, and archive text, image, and video measurements on the
local Mac. Results remain local unless the user explicitly chooses to share.
The consent flow previews the exact destination and payload digest, and a
durable receipt prevents accidental duplicate submissions.

Better model and memory decisions

CLI, Server, and Desktop now consume one atomic model registry and
recommendation policy. Per-model Metal limits are tuned from the machine's
available unified memory, and rejected launches return actionable fit guidance
instead of failing deep in model loading. New per-model metrics expose request,
token, latency, and throughput totals without mixing aliases.

Desktop quality-of-life improvements

• Opt-in persistent memory can retain useful long-term context across
  conversations and remains editable or removable in Settings.
• Mermaid code blocks render as isolated, network-blocked previews.
• Dictation overlaps speech-model page-in with the utterance to reduce the
  perceived first-use delay.
• An update card explains whether the installed app is current, behind, or
  ahead of the public release without forcing an update.

Experimental performance routes

Qwen3.8 Flash-Next gains an opt-in fused single-token GDN kernel. On the
qualified M3 Ultra run it improved one complete GDN layer by 26.28% and
end-to-end decode from 25.43 to 27.04 tok/s (+6.35%), with bit-identical
tokens and recurrent state in the qualification workload. It remains disabled
by default while broader hardware evidence is collected. Qwen3.8 27B also adds
an explicit experimental DFlash2 route and a conservative prompt-lookup route;
unsupported requests continue to fail closed or use the ordinary path.

Reliability and operations

• Explicit quantized-KV and paged-cache requests now reject unsupported cache
  layouts before Ready instead of starting healthy and failing later.
• Paged prefix reuse commits hit metrics only after successful reconstruction
  and rolls back block ownership on failure.
• Singleton MTP keeps exclusive scheduler ownership until its request departs,
  so concurrently arriving plain or tool requests wait instead of aborting
  both streams with a 503.
• A documented LaunchDaemon deployment supports qualified headless macOS
  servers while keeping the same user-owned cache and service boundaries.
• Model-download, GUI attachment, update, video-job, and MTP lifecycle failures
  have more bounded cleanup and clearer recovery behavior.

Upgrade

pip install -U rapid-mlx

Homebrew follows the PyPI release through its normal autobump process. Desktop
users receive the signed and notarized update through the production appcast.

All changes

• chore: bump version to 0.13.4 (711da18)
• fix(mtp): isolate singleton verifier admission (#2964) (f05d8d5)
• feat(benchmark): add consent-gated atomic uploads (#2931) (3b1a553)
• fix: reject explicit quantized KV pre-ready on unsupported families (#2954) (#2959) (9106a36)
• fix(cache): fail closed and transact paged prefix reuse (#2960) (1a5c6bd)
• feat: add local-first Community Benchmark workspace (#2882) (3fe7087)
• feat(video): add LTX 2.5, Wan 2.1, and CogVideoX-Fun to Desktop (#2947) (6096094)
• docs: qualify headless macOS service deployment (#2952) (016a16d)
• catalog: consume atomic recommendation policy (#2871) (87d8825)
• feat(markdown): preview Mermaid diagrams securely (#2812) (86dbcde)
• docs: refresh 0.13.4 large-model benchmarks (#2942) (65a0fbc)
• [codex] Add lightweight desktop update discovery card (#2919) (c9aea88)
• Expose per-model request performance metrics (#2915) (f7ea5bc)
• test(mac): wait for authoritative drop completion before retry (#2938) (ede7158)
• fix(mtp): preserve requests on admission failure (#2934) (bcf8e51)
• feat(catalog): add atomic product model registry (#2857) (97755ce)
• feat(mac): add experimental Video workspace (#2844) (615a8c5)
• fix(doctor): keep runtime probe within five-second contract (#2926) (2a0873f)
• Qualify Qwen3.8-27B DFlash2 as an explicit experimental route (#2920) (abf6a93)
• perf(qwen4): fuse single-token GDN decode (#2918) (4b35ed0)
• Build Desktop video serving foundation (#2810) (63dac4f)
• Fix doctor runtime dependency health (#2865) (7e86e04)
• perf(qwen4): qualify conservative prompt lookup (#2911) (087826f)
• test(mac): retry only unaccepted attachment drags (#2909) (d8b2890)
• fix(mac): overlap dictation model page-in with the utterance itself (#2905) (3024358)
• feat(mac): offer one-click GitHub star via gh CLI (#2772) (01739b3)
• fix(tests): pin prefix-eviction stub to the utilization-ratchet generation (#2907) (d8ffed0)
• feat(proto): define atomic model runtime and benchmark contracts (#2840) (9b50840)
• feat: auto-tune per-model Metal memory limits and return actionable admission errors (#2880) (82b703a)
• fix(ci): use supported recovery for dequeued PRs (#2896) (6c61646)
• fix(ci): follow model memory confirmations in GUI flows (#2826) (612aad4)
• feat(mtp): default verified artifacts to continuous batches (#2875) (1869722)
• fix(ci): allow dequeued PR label recovery (#2893) (813112e)
• fix(ci): recheck stale transcript jump element (#2890) (1faaa7c)
• feat(mtp): adapt continuous batching for Qwen3.8-27B (#2854) (85b5e21)
• fix(ci): requeue dropped heads through ready labels (#2881) (51e6791)
• feat(mtp): add continuous speculative batching foundation (#2842) (644afee)
• feat(mac): persistent memory extraction from completed conversations (#2850) (275b10f)

Install: brew upgrade rapid-mlx or pip install -U rapid-mlx==0.13.4 (or just rapid-mlx upgrade).