What's new in v0.14.1
Rapid-MLX 0.14.1
Rapid-MLX 0.14.1 makes long-document work and multimodal chat more dependable,
while adding stronger regression protection for sampled MTP decoding. It also
records an intentionally negative large-model qualification so users can see
why an expensive checkpoint was not added to the product catalog.
Large and scanned PDFs
Desktop can now analyze large selectable PDFs, image-only scans, and documents
that mix selectable and scanned pages. Extraction runs behind a bounded cache,
document reads use hard deadlines, and follow-up questions can retrieve
specific outlines, pages, or sections instead of forcing the entire file into
one prompt. (#3292)
The product-path dogfood covered both ends of the workflow:
| Document | Verified result |
|---|---|
| 300-page selectable Chinese PDF | 125,833 characters cached; all 300 pages completed; 30 chapter rows resolved to real page and character offsets |
| 6-page image-only PDF | OCR text reached the model and multi-turn synthesis decoded at 45 tok/s |
| Mixed selectable + scanned PDF | Scanned pages were retained and the extraction reached an honest complete state |
The same change closes several failure boundaries: OCR/render failures no
longer masquerade as complete extraction, progress heartbeats cannot extend a
read forever, removal races cannot publish an orphaned document, invalid IDs
are bounded, and a model that keeps asking for tools is forced into synthesis.
Multimodal chat stops runaway repetition safely
An exact token loop in a tool-free multimodal request could previously continue
until Metal exhausted its resource handles, leaving the client waiting for an
empty HTTP 500. The scheduler now evaluates the existing conservative detector
every eight generated tokens, retires the stopped batch row immediately, and
returns the valid partial response with a normal length finish.
(#3291)
The reported 61-token cycle is detected at token 184. The focused lifecycle,
batching, repetition, metrics, and process-loop validation passed 180 tests.
Sampled MTP has a stronger correctness net
New distribution-level tests cover independent speculative acceptance draws
and rejection-residual sampling, and real-weight checks force K=2 and K=3
paths. (#3294)
This guards a subtle false-green boundary: replacing rejection-residual
sampling with target-distribution sampling passed all 246 tests in the prior
focused MTP suite, while four of the five new distribution checks failed and
exposed total-variation error up to 0.14. Real-weight smoke recorded 118 K=2
and 147 K=3 speculative attempts, so the deeper arithmetic is exercised rather
than inferred from a nonzero acceptance rate.
An honest 256 GiB model qualification
The repository now contains isolated tooling and reproducible evidence for an
experimental DeepSeek V4.1 Flash REAP 2-bit checkpoint. This is not product
model support: the runtime is outside package discovery and there is no catalog,
Server, or Desktop entry. (#3301)
The measurements below used an Apple M3 Ultra with 60 GPU cores, 256 GiB of
unified memory, and MLX 0.32.2.
| Measurement | Result |
|---|---|
| Artifact tensor size | 212,930,051,680 bytes (about 199 GiB) |
| Strict resident load | 239.44 s |
| Peak MLX memory | 213.51 GB |
| Conservative decode | 7.31 tok/s |
| Best short-decode ceiling | 7.92 tok/s |
| Rapid-MLX product floor | 12 tok/s |
The checkpoint preserved 99.9949% mean robust routing mass and passed tiny-
configuration parity within 1.4e-6 with 100% argmax agreement, but speed missed
the product floor by 34.0%. The checkpoint is
archived separately for expert reproduction,
but remains explicitly experimental and outside the product catalog rather than
becoming a catalog-managed download.
Upgrade
pip install -U rapid-mlxHomebrew follows the package release through its normal autobump process.
Desktop users receive the signed and notarized update through the production
appcast.
All changes
- chore: bump version to 0.14.1 (ef9f1cb)
- bench: qualify native 2-bit V4.1 REAP artifact (#3301) (cc8681a)
- Large & scanned PDF analysis (restack of #1975 + review fixes) (#3292) (1c6e18e)
- test(mtp): behavioural coverage for the sampled MTP path (from #2140) (#3294) (72b6c0c)
- fix(mllm): stop runaway repetition before Metal exhaustion (#3291) (e86a6a9)
Install: brew upgrade rapid-mlx or pip install -U rapid-mlx==0.14.1 (or just rapid-mlx upgrade).