v0.13.1

What's new in v0.13.1

Rapid-MLX 0.13.1 focuses on safer model changes and smoother first-use paths:
assistant replacement is now an explicit transaction, large multi-variant
repositories can download only the requested format, and mirrored models stay
on the fast download route.

Highlights

Experimental Qwen3.8-Flash-Next text inference — The M1 lane supports the
mixed 4-bit checkpoint through the qwen3.8-flash-next-4bit alias on Macs with
at least 128 GB of unified memory. This release exposes the experimental text
lane through the CLI and API; release validation used a 256 GB Mac, and Desktop,
MTP, and vision support remain planned for a later release.
(#2433)

Switch assistants without losing track of active work — API model
replacement now has explicit reject, wait, and abort policies shared by the
engine and server; Desktop uses the safe reject policy while a request is busy.
Request ownership transfers atomically, admitted work is quiesced before reload,
and auxiliary dictation residency remains independent of the selected assistant.
The /v1/models/load API defaults memory_policy to evict_first_if_needed:
it keeps the rollback-safe load when old and new fit together, evicts the
replaced assistant first only when keeping both breaches the limit but the
replacement alone fits, and otherwise returns a typed 507 with
replacement_projection before destructive mutation. Set
memory_policy=keep_then_commit to require rollback-safe loading.
(#2369)

Download one usable variant instead of an entire repository — rapid-mlx pull accepts --bits or --format for repositories that store several
quantizations side by side. This avoids turning a small-model download into tens
of gigabytes of formats the user did not select. (#2145,
#2338)

More dependable downloads and tool routing — Subfolder-based quantizations
can use the Rapid mirror instead of being hard-routed upstream. Dedicated
evaluation scenarios now catch explicit current-weather prompts that choose a
generic search fallback instead of the advertised Weather tool. (#2279,
#2222,
#2327)

Photo input uses the validated bundled vision runtime — The Desktop
sidecar carries the updated vision stack and the release gate now proves that
two cached vision families produce image-dependent answers before an artifact
can ship. (#2384,
#2380)

Vision-capable Qwen3.5, Qwen3.6, and supported Qwen3.8 checkpoints use the
vision lane automatically when mlx-vlm 0.6.16 or newer is installed and the
checkpoint meets its measured vision-memory recommendation. If automatic
admission cannot use that lane, serving falls back to text with a
machine-readable reason. The experimental Flash-Next checkpoint is explicitly
text-only in this release. CLI users can force text serving with --no-mllm,
and Desktop exposes the same per-model Performance choice. Requests that enable
MTP or another speculative decoder select the text lane, where that decoder is
supported.

A better first model for each Mac — First run now prefers an eligible
cached model, then chooses a starter that fits the machine instead of applying
one recommendation to every Mac. Supported hybrid vision checkpoints are
routed to their working multimodal lane, so photos work with the recommended
model without a manual engine choice. (#2385,
#2219,
#2388)

Reliability and setup corrections

• API request validation accepts either a scalar or array stop, rejects
  invalid timeouts, requires real JSON booleans for resident-model controls,
  and reports the precise invalid field path. (#2367,
  #2371,
  #2372)
• Guided setup now says explicitly that it preserves user data, avoiding a
  misleading destructive-action warning. (#2239,
  #2322)
• The one-time telemetry consent question waits until Rapid has delivered a
  successful chat reply, dictation transcript, or generated image instead of
  interrupting first launch. Nothing is sent before explicit consent.
  (#2424)
• Share CLI coverage now exercises only supported invocation forms and runs in
  CI. (#2377,
  #2382)
• rapid-mlx doctor distinguishes the executable running inside the active
  environment from another installation found on PATH, and explains how to
  resolve a mismatch. (#2352,
  #2402)
• Serving an uncached model while offline stops with one actionable cache and
  connectivity explanation instead of retrying the same download or suggesting
  an unrelated serving lane. (#2357,
  #2423)
• 8 GB Macs start with a model that fits. First run recommends
  lfm2.5-1b-4bit on the lowest memory tier, avoiding the RAM warning produced
  by the previous 2.6B starter. (#2432)
• Dictation stays resident and is restored after switching chat models.
  (#2400)
• MTP generation remains bound to its worker when another model loads, so the
  original assistant continues answering instead of losing its GPU stream.
  (#2441)
• Model-switch memory estimates credit the assistant being replaced and use
  the same fit thresholds as the actual replacement transaction. Desktop's
  low-memory threshold is now 100% instead of 85%: projections from 95–100%
  are advisory, while values above 100% require confirmation. The earlier
  conservative threshold is superseded because the projection now credits the
  model being replaced, avoiding a warning for memory that will be freed before
  the new model starts.
  (#2443,
  #2444)
• Speech to Text automatically arms after model lifecycle changes when the user
  has enabled it; there is no extra manual start button to press.
  (#2448)
• JSON-schema responses and bounded-thinking or title-generation requests now
  terminate normally on the vision lane: their request-local output processors
  and complete configured EOS set reach the multimodal scheduler, while photo
  requests continue using the same server. (#2471)
• Experimental Qwen3.8-Flash-Next remains on its supported text lane; MTP and
  other speculative decoding requests also choose the text lane; and
  text-diffusion assistants can replace another assistant without a spurious
  group-conflict response. (#2472)
• iPhone HEIC photos work through picker, drop, and paste. Supported still
  images are normalized to truthful JPEG or PNG bytes at the shared attachment
  boundary while preserving the attachment size limit.
  (#2467)
• High-resolution photos are downscaled before Desktop sends them so they fit
  the model's vision budget, and a typed image rejection no longer leaves a
  failed attachment turn that contaminates the conversation.
• Launch no longer rehashes every sealed app resource merely to choose the
  Keychain namespace, and Escape cannot silently decline the post-value
  telemetry-consent invitation. (#2470)
• A rejected photo no longer contaminates the next text turn, and multimodal
  unload/reload cannot reuse a stale generation worker. (#2379,
  #2378,
  #2401,
  #2397)
• Readiness and connection guidance now report the live port, selected model,
  and served name. Public streaming request IDs can be cancelled reliably.
  (#2386,
  #2348,
  #2395,
  #2353,
  #2398,
  #2342)
• Failed primary reload recovery keeps registry, residency, routing, readiness,
  and speech handoff consistent. (#2394,
  #2360)
• Downloads retain the catalog's size warning when remote metadata is missing,
  and cached pulls are reported as verified rather than newly downloaded.
  (#2391,
  #2350,
  #2392,
  #2349)
• Complete cached Kokoro and Whisper Turbo snapshots are recognized as runnable
  using their verified audio weight layouts. (#2406,
  #2417)
• System-prompt settings distinguish the saved instruction from the effective
  prompt and avoid touching optional credentials until they are needed.
  (#2341)
• A complete offline cache with one immutable snapshot remains runnable even
  when a branch ref is unavailable; ambiguous multi-snapshot caches remain
  unresolved rather than guessing. (#2351,
  #2404)
• Server discovery for model switching uses live socket ownership instead of
  scanning mounted filesystems, avoiding unrelated volume-access prompts.
  (#2343,
  #2408)
• Native file URL drops on the chat composer use the existing attachment
  importer, and unsupported drops cannot insert a local path into the message.
  Contributed by osdodo.
  (#2396)

Release evidence and corrections

• Corrected the published Nemotron Labs Diffusion description so it accurately
  describes the supported text-model path. (#2376)
• Recorded a reproducible M2 Pro performance comparison between the published
  0.13.0 and 0.12.18 packages. (#2375)

Known issues

• Suffix decoding with sliding-window models such as Gemma 4 and GPT-OSS can
  abort a request at a window boundary. Disable suffix decoding for these
  models as a workaround. (#2463)

All changes

• chore: bump version to 0.13.1 (#2482) (819db66)
• docs: stage 0.13.1 release notes (#2393) (3e0bc30)
• release: integrate 0.13.1 train 6d (#2479) (c730e45)
• release: integrate 0.13.1 train 6c (f911af1)
• release: integrate 0.13.1 train 6b (2063517)
• release: integrate 0.13.1 train 6 (#2449) (aaf2e01)
• ci: union Apple MLX coverage into changed-lines gate (#2440) (a6b820c)
• fix(mac): use the safe 8 GB Quickstart model (#2432) (aaa2b8a)
• release: integrate 0.13.1 train 5 (62a038c)
• release: integrate 0.13.1 train 4 (19616c2)
• release: integrate 0.13.1 train 3 (32288e3)
• release: integrate 0.13.1 train 2 (eb9b335)
• release: integrate 0.13.1 train 1 (854d049)
• test(lifecycle): cover cancelled sibling retirement (36265f5)
• fix(lifecycle): propagate post-commit cancellation (cb837dd)
• test(residency): cover grouped reload transaction (286acfc)
• fix(residency): quiesce admitted requests before reload (84bb0f1)
• test(lifecycle): assert committed stop-failure routing (8b636bb)
• fix(lifecycle): commit handoff before engine retirement (7034584)
• fix(residency): validate explicit replacement groups (838855e)
• test(lifecycle): close exact-head validation gaps (361db6b)
• fix(server): materialize assistant before lifecycle abort (9ccc6d4)
• fix(server): preserve lifecycle abort reason in SSE (d8cc848)
• test(server): pin dictation across assistant transitions (28da6b7)
• fix(lifecycle): preserve precommit abort ownership (ae1a05c)
• fix(lifecycle): close replacement convergence gaps (6ab3eac)
• fix(server): wait for resident leases before replacement (7c935dc)
• fix(server): translate route-bound lifecycle aborts (bcd11ce)
• fix(server): reject busy replacement before eviction (3dfd886)
• fix(engine): transfer non-stream admission on commit (d2daf07)
• fix(engine): publish scheduler requests atomically (ee3f184)
• fix(server): surface pre-scheduler replacement aborts (4e30984)
• fix(engine): abort pre-scheduler admissions (527a2a6)
• fix(engine): preserve concurrent admission ownership (5d0d9b6)
• test(server): lock dictation coexistence during replacement (d8552b7)
• test(server): satisfy lifecycle type gate (073009a)
• feat(server): add explicit assistant replacement policy (7470607)
• docs(perf): compare v0.13.0 with v0.12.18 on M2 Pro (5f8984d)
• feat(pull): select one variant of a multi-variant repo with --bits/--format (#2145) (#2338) (43afedb)
• docs: correct Nemotron 0.13.0 wording (df31169)
• test(evals): lock explicit-weather routing to the weather tool (#2222) (#2327) (30be6a1)
• test(mirror): cover subfolder filtering in Linux lane (1c81f80)
• fix(mirror): serve subfolder-per-quant repos from R2 instead of hard-declining (bc45286)

Install: brew upgrade rapid-mlx or pip install -U rapid-mlx==0.13.1 (or just rapid-mlx upgrade).