MTPLX 2.9.1

MTPLX 2.9.1

Agent coding sessions run to completion: long-context crash fixes, no hidden output caps, reasoning preserved across turns, and a built-in flight recorder for diagnosing any session.

Engine

• Fixed: agent sessions could truncate and crash near 19,000 tokens (#310). The paged KV cache derived its capacity from a stompable claim instead of the pages it had actually allocated. Long coding sessions now run to the model's full advertised context.
• Fixed: shutdown segfault (#303). The daemon parks its model-owner thread and clears MLX streams at exit, so quit and restart are clean.
• Turbo profile truth. 2.9.0 shipped one turbo fast-path flag that was runtime-dead, so turbo did not apply its full intended configuration. The fast-path environment is now a single shared block, /health reports exactly what the profile set, and a per-lane kernel selfcheck runs at startup. If you benchmarked turbo on 2.9.0, re-run it.
• Multi-turn cache reuse holds at scale. All encode paths now share one tokenization policy, so warm agent turns no longer hit cache walls at assistant reasoning boundaries; tool-call turns bank their just-generated output directly from live KV with no GPU recompute (follow-up turns restore the full prior context at exact length); interrupted background commits retry instead of silently giving up.
• The model no longer re-derives its own reasoning on long sessions. When a client echoes prior reasoning back, MTPLX renders it for turns its committed cache has not yet covered instead of an empty scaffold. In live sessions this ended a failure mode where one marathon turn re-thought a 57,000-token derivation from scratch.
• The model pack owns draft sampling. Stamped draft-sampler settings win over stale client-side pins, so speculative decoding runs the configuration each pack was tuned with.

Agent clients

The app and mtplx start now write identical client configurations for every supported coding agent, and mtplx doctor reports the truth about each lane.

• OpenCode: uncapped generation by default — the managed plugin strips exactly OpenCode's injected 32,000 output ceiling while explicit caps pass through untouched. Reasoning and reasoning-effort selection are honored end to end, prior reasoning round-trips across turns, and each session carries a stable cache identity.
• Pi: the reasoning-effort picker works and maps to the loaded model family's levels. The real output ceiling is advertised (Pi silently applies 16,384 when a model's metadata omits it), and a managed extension strips Pi's generated default cap and adds per-session cache identity.
• Hermes: requests now carry client identity and the configured reasoning effort (agent.reasoning_effort), and the server strips Hermes's injected 65,536 default cap — Hermes cannot express "no cap" on its own. Explicit user caps are honored on every lane.

Flight recorder and mtplx trace

Every request now records a per-second flight log: tokens/sec, context growth, speculative acceptance by depth, verify/draft time split, prefill, and the final outcome — including cancelled and disconnected requests, which previously left no trace.

• GET /v1/mtplx/flight answers "is it hung or thinking?" while a request runs: phase, live tok/s, acceptance, stall age, and the tail of the text being generated.
• mtplx trace turns any coding session into a diagnosis: per-turn timelines joined to your OpenCode history, cache-reuse analysis, automatic pathology flags, repetition autopsies, and a self-contained HTML report per session.
• Recording is local-only JSONL under ~/.mtplx/metrics — a few MB per day of heavy use, capped at 256 MB by rotation. Set MTPLX_FLIGHT_RECORDER=off to disable it.

Updating

• App: Sparkle offers 2.9.1 automatically, or download the DMG at mtplx.com.
• CLI: pip install -U mtplx or brew upgrade mtplx.

MTPLX 2.9.0

MTPLX 2.9.0

Faster decode, smooth streaming at any context length, smaller model packs, and built-in model updates.

	2.8.3	2.9.0
Decode throughput	baseline	15 to 20% faster typical, up to 60% on code-heavy output
Visible stream freezes (8k context, ~100 tok/s)	102 per session	5
Worst streaming stall (minified JSON)	725 ms	109 ms
CPU while streaming	26 to 28%	18 to 23%
CPU with Settings open while streaming	82 to 97%	30 to 37%
Qwen 3.8 pack downloads	15 to 21 GB	0.4 to 0.6 GB smaller per pack
Updating a pack you already have	full re-download	240 to 450 MB

Engine

• Decode is 15 to 20% faster than 2.8.3 on typical workloads, combining the decoder work below with the re-quantized draft heads. Code-heavy generations measured up to 60% faster in paired same-machine runs.
• Streamed output is released on token boundaries and grouped into fixed-cadence rounds. Long streams no longer freeze and then dump text: 102 visible freezes per 8k-context session is now 5, and the worst stall dropped from 725 ms to 109 ms.
• Engine CPU while streaming dropped from 26 to 28% down to 18 to 23%.
• The daemon stops rebuilding its full dashboard payload ten times a second when nothing is running. Idle means idle now, and live metrics are unchanged while a request is active.
• The depth tuner warms every candidate before its timed rows. It used to pay model load and kernel compile inside the first measurement, which penalized deeper depths and made it save shallow configs that lost to a static depth 3 (#271).
• If you consume raw SSE and want every token the instant it decodes, set MTPLX_STREAM_COALESCE=0.

Models

• Every mtplx pull now records which revision it downloaded and pins the download to that exact commit. A pack on disk can no longer silently differ from what is published.
• New: mtplx models --check shows the update state of every cached pack. mtplx models --update <repo> syncs one, downloading only the changed files: typically 240 to 450 MB instead of a 15 to 21 GB re-pull.
• Works for packs downloaded by older MTPLX versions too. When the check cannot prove your copy is current, it says unknown instead of guessing.
• All six Qwen 3.8 packs re-shipped with quantized speculative-decoding draft heads: 4-bit heads on 4-bit trunks, 8-bit on the 8-bit trunk, FP16 variants for M1/M2 contain no BF16 anywhere. Trunk weights are unchanged, so upgrading rides the delta updater.
• Same answers, verified: every pack ran a paired multi-seed acceptance battery against its previous head and shipped only flat-or-better at every speculation depth. The Quality FP16 head reproduces its source head token for token on every gated run.

Pack	Draft head	Download	Decode at depth 3
Optimized Speed	INT4/g64	610 MB smaller	46.8 tok/s, 2.3x plain decode
Bare Speed	INT4/g64	610 MB smaller	49.9 tok/s, 2.3x plain decode
Optimized Quality	INT8/g64	398 MB smaller	39.2 tok/s, 3.0x plain decode
Speed FP16 (M1/M2)	INT4/g64	610 MB smaller	45.4 tok/s, 2.3x plain decode
Bare FP16 (M1/M2)	INT4/g64	610 MB smaller	50.2 tok/s, 2.3x plain decode
Quality FP16 (M1/M2)	INT8/g64	398 MB smaller	48.7 tok/s, 2.8x plain decode

App

• The model picker shows when a cached pack has an update, with its download size. One click updates it. If the model you are running was updated, you get a restart prompt.
• Rendering a live stream now costs the same whether the message is 10 lines or 10,000. CPU with Settings open during a stream dropped from 82 to 97% down to 30 to 37%.
• Long code streams flow inside the message with no nested scrollbars and no sideways travel. Code wraps, grows live with its content, and hands off cleanly to the final syntax-highlighted view. Verified on a 42,000-token generation.
• Streamed Markdown stays visually stable. Prose no longer reflows or flashes as lines arrive, and code keeps its colors while it grows.
• Reasoning streams as append-only text in the order the model produced it. No more briefly duplicated or reshuffled thoughts.
• Forged models are first-class: your own Qwen 3.8 builds report the right capabilities (vision included), Use Now puts them straight into the model picker, Forge profile metadata is honored, and verification is tied to the exact artifact you built.

Fixes

• Switching models while the daemon was degraded could hang forever. It restarts cleanly now.
• Fans no longer stay pinned at max after a failed startup.
• Pressing Return to send no longer drops the first characters of fast typing. Shift+Return inserts a newline.
• The status dot now says why the daemon is degraded instead of only changing color.
• Catalog download sizes were re-audited against the live repos and corrected.

Updating

• App: Sparkle offers 2.9.0 automatically, or download the DMG at mtplx.com.
• CLI: pip install -U mtplx or brew upgrade mtplx.
• After updating, run mtplx models --check to pick up the smaller packs.

MTPLX 2.8.3

MTPLX 2.8.3

Released 2026-08-18. Follows 2.8.2.

Why this patch

2.8.2 fixed the start wizard's re-tune loop, but field reports of slow,
stuttering chats kept coming — and they were right. Two more regressions
shipped in 2.8.0 and survived until now because both live on a code path
release QA never exercised: the uncapped request, which is what every
desktop, web, and agent chat actually sends. Benchmarks and gates always
cap max_tokens, and capped requests take a different path.

The streaming freeze

2.8.0 added a wire safeguard for the runaway-loop trimmer: while an
uncapped request is armed, the stream held a fixed ~448-token tail so a
trim could never chase bytes already delivered. At chat speeds that
silenced the wire from roughly token 320 to token 768 on every
uncapped response: reasoning froze for 6–11 seconds while the speed
readout collapsed, then the backlog arrived in one burst — and the last
~448 tokens of every response arrived the same way. The engine was
decoding normally the whole time, which is why every server-side number
looked healthy.

The holdback is now engaged only while the output actually shows a
forming loop. Healthy responses stream live, byte for byte, exactly like
2.7.1. A genuine runaway loop still gets trimmed with at most a short
prefix of the repeated run ever reaching the wire — still strictly
better than 2.7.1, which streamed all of it.
MTPLX_REPETITION_STREAM_HOLDBACK=candidate|strict|off selects the new
default, the 2.8.0–2.8.2 behavior, or the pre-2.8 wire.

The boot burn

2.8.0's turbo profile walked a background warm ladder up to 32,768-token
prefills after every boot so deep-context benchmark rows would start
warm. On real machines that meant 30–60+ seconds of full-throttle GPU
after every launch (2.7.1: about 4.5 s), warm rungs re-firing between
chat turns, and a chat sent mid-rung waiting several seconds for its
first token.

The product ladder is back to the two rungs interactive chat actually
reaches early. Benchmark harnesses that want deeper buckets pre-warmed
opt in with MTPLX_WARMUP_LADDER. Background warm steps now also wait
for 90 seconds of request quiet (MTPLX_WARMUP_IDLE_GRACE_S) before
touching the model, so warming never competes with a conversation.

The app was the other half

Founder testing at temperature 1.0 on a long, multi-turn conversation
caught what a fresh chat never showed: the desktop app itself rendered
in O(entire transcript) per frame. The window's SwiftUI hosting view
re-measured every realized message on every constraint invalidation
(a third of the main thread at idle, up to 62×/s while streaming — the
guard written to prevent exactly this had silently never applied), the
markdown pipeline re-counted every character of every block per frame,
and per-delta paths copied the whole answer to ask if it was empty.
When those walks stalled a frame, the typewriter's catch-up path pasted
the whole backlog at once — the literal freeze-then-vomit. The engine
was streaming cleanly the entire time; replaying the wire gate over the
founder's exact 37k tokens showed zero holdback engagements.

2.8.3's app build fixes all of it: the transcript min-size walk is
dead, fence classification and syntax-lex state are computed once per
block instead of once per frame, scroll pacing no longer invalidates
the view tree, catch-up is rate-limited to read as fast typing (never a
paste), the 10 Hz metrics chip no longer re-evaluates every bubble or
burns a core parsing its stream byte-by-byte, and markdown tables no
longer draw rows on top of each other (cells measured single-line but
drew wrapped).

QA changes

The release pillar gate now includes an uncapped streamed chat measured
for delivered cadence — any gap over 2 seconds fails the release. The
capped-only blind spot that let both regressions through is closed.

App-side, the bar moved too: streaming QA runs on a heavy multi-turn
conversation (fresh chats hide every O(transcript) render term), and
"smooth" is judged at three layers — engine wire, app ingest telemetry,
and the actual on-screen text — because the first two were green while
the third was frozen.

Measured on an M5 Max, 27B Bare-Speed, uncapped chat

	2.8.2	2.8.3
Worst mid-stream gap	6.4–11.5 s, every response	under 1.3 s
Time to first token (warm daemon)	up to 6.6 s under warm-rung contention	0.26 s
Boot warm burn	30–60+ s max GPU	~4.5 s
End-of-response burst	~448 tokens at once	none

Round two: what the founder's retest caught

The first 2.8.3 candidate fixed the measured pathologies and then
failed a human in three ways the instruments weren't pointed at. This
round is those three, fixed and re-measured:

• Blank transcript mid-generation — the candidate's lazy transcript
  culled every visible row under the app's own scroll driver. Eager
  again; streamed tables (wrapping cells included) verified in pixels
  while arriving.
• Thinking well rewriting itself — reasoning now renders as plain
  text, and the live ticker's window anchors at line breaks so a
  rendered thought line never changes after you've read it.
• Freeze-then-burst cadence — server-side: whitespace-free runs
  flush incrementally, token emission precedes cache housekeeping
  barriers, and the auth middleware no longer buffers stream frames.
  Same-prompt A/B: ~30 sub-second delivery silences per answer → single
  digits; generator gaps over 200 ms → zero; every request now records
  a producer gap census so this regression class can never ship silent
  again.

Round three: the stutter that only humans could see

The founder retested with his own hands and the stutter was still
there — while every hands-off verification pass stayed clean. Both
observations were correct, and the difference between them was the
bug:

• Interaction starved the UI's layout guard. The guard that stops
  the window from re-measuring the entire conversation ran in a
  run-loop phase macOS skips while input events keep arriving. Touch
  the mouse and the guard stopped running; every layout pass walked
  the whole transcript and screen updates coalesced into visible
  freezes. Phase-aligned A/B with synthesized human input on the same
  build and prompt: hands-off was clean on both binaries; 40 s of
  wheel-scrolling went from 70 stalls / 18.7 s frozen / worst 1.31 s
  to one 197 ms stall, and 30 s of cursor movement went from
  91 stalls / 26.7 s frozen to zero. The guard now runs every
  run-loop turn, input storms included.
• Auto-follow fought the user. Scrolling up mid-generation raced a
  synchronous bottom-pin whose user-scroll signal was set
  asynchronously; momentum and classic wheel mice weren't covered at
  all. The pin now yields to any user scroll instantly and re-engages
  only at the bottom.
• Cancelled runs were unmeasured. The founder's stutter report came
  from a run he cancelled — and cancelled requests logged no stream
  census at all. They now log the full producer gap census and sliding
  windows for the streamed portion.

About multi-turn TPS

Follow-up turns in a conversation decode slower than the first — the
founder measured 43 → 40 → 33 tok/s across three turns at under 10k
context — and this release deliberately does not paper over it. Two
real mechanisms, both now precisely attributed in every request
record: each verify cycle costs more as context grows (~60 ms/cycle at
1k context → ~77 ms at 8k, honest attention physics plus a
context-linear draft cost), and speculative acceptance tracks content
entropy — at temperature 1.0 it collapses in free-form prose (down to
~0.4 by depth 1 in wrap-up passages) so each cycle commits fewer
tokens. Neither is a regression: the same curves are measurable in
every 2.8.x build. The per-cycle cost work is the standing decay
track, continuing in 2.8.4; acceptance-vs-entropy is the 3.8 MTP head
calibration campaign. What 2.8.3 ships is the instrumentation that
makes both visible per-request instead of anecdotal.

MTPLX 2.8.2

Why a second patch today

Within hours of 2.8.1, two independent reports showed the same failure: load a model, and with no requests at all the machine heats up while API requests never arrive (#280, and a report on X). If that happened to you, 2.8.1's start wizard was silently re-running the multi-minute tuning benchmark on every launch — with the API port still closed while it ran. 2.8.2 fixes that loop and everything found underneath it. If you updated to 2.8.0/2.8.1 today: update again, it takes one minute, and your saved tune results now stick.

Fixed

• mtplx start no longer re-offers tuning on every launch. The tune record's lookup key and its save key had drifted apart in 2.8.0 (the per-model profile work updated one side only), so the wizard could never find the record it had just saved. Accepting the re-offer meant minutes of maxed GPU with the server not yet listening — the reported "heats up + API unreachable" pair. One shared key constructor now, with a regression test that fails if the two sides ever drift again.
• The wizard tunes the model you actually picked. Picking a non-default model tuned the machine's default instead (picking the 4B tuned the 27B) and saved a record the launched model could never use.
• A picked local model is never silently swapped for the default. LM Studio-style folders whose name matches the verified default were re-routed to the Hugging Face repo id, producing "Model is missing. Download?" for a model already on disk (#279).
• Idle daemons no longer hammer session-bank/manifest.sqlite. Every health/dashboard poll opened the SSD-cache manifest and ran a full-table aggregate (~8 sqlite opens per poll, continuously visible in Activity Monitor, #280). Steady-state polling now costs at most one manifest read per 5 seconds.
• mtplx --version reports the right version again. The published 2.8.1 wheel identified itself as 2.8.0 (2.8.1); 2.8.2 prints 2.8.2 (2.8.2) — worth checking after you update.

Nothing on the generation path changed: decode, prefill, sampling, tool calling, and the session cache behave exactly as in 2.8.1 (verified flat-or-better in interleaved A/B before ship).

MTPLX 2.8.1

MTPLX 2.8.1

Released 2026-08-17. Hotfix on top of 2.8.0, published the
same morning.

Why a patch an hour after 2.8.0

2.8.0's headline work made long agent sessions commit their cache
frontier on every turn, including turns that carry images. Our release
pipeline runs a correctness sentinel for exactly that surface: send a
transcript with one image, then repeat the identical transcript with a
different image, and require that the second request never reads cached
state from past the image position. That sentinel failed on the desktop
build gate.

The cause: the session frontier is keyed by token ids, and every image
placeholder shares a single id, so two different images look identical
to it. Once vision histories started committing, a repeated transcript
with different pixels could restore the previous image's KV wholesale
and answer about the wrong image.

The published PyPI wheel for 2.8.0 carried this defect for about an
hour. The desktop DMG never shipped with it; the gate blocked it.

What changed

Vision conversations keep their full prompt-cache behavior through the
content-keyed store introduced in 2.8.0: it derives cache keys from the
actual image bytes, so identical pixels restore the whole prefix and
different pixels can never match past the first image token. What
changed is the raw-id session frontier: image-bearing histories no
longer advance it, on any of its four write paths. Same speed for the
honest case, hard stop for the aliasing one.

Five new regression tests pin the behavior, and the release gate that
caught it now runs against a hermetic cache directory so a previous
run's state can never mask or fake a result.

Upgrading

• PyPI: pip install -U mtplx (2.8.1)
• Homebrew: brew upgrade mtplx
• Desktop app: 2.8.1 (build 2008001) via Sparkle or the website DMG

Nothing else changed from 2.8.0; its release notes remain
the reference for what is new.

Known cosmetic quirk in this wheel

The mtplx --version banner in the 2.8.1 wheel reads mtplx 2.8.0 (2.8.1): the display constant missed the hotfix bump. The package
version, pip show, and all behavior are correct, the desktop build
displays 2.8.1, and the banner is fixed on main. Not worth a third
release in one morning.

MTPLX 2.8.0

Use 2.8.1 instead.
The 2.8.0 wheel carried a vision session-cache defect for about an hour
after publish: a repeated transcript with a different image could restore
the previous image's cached context. Our release gate caught it before the
desktop build shipped; 2.8.1 is this release plus that fix.

MTPLX 2.8.0

This release is about trust. Over the last week people started benchmarking
MTPLX seriously and running long agentic sessions against it, and both groups
found real problems: sessions that quietly stopped reusing their cache past
38k tokens, stats that could disagree with what the engine actually did, and
a first-run path with sharp edges. 2.8.0 closes 45 commits of that work. The
rule for the whole release was simple: every number the server reports must be
the number the engine lived, and every documented path must work on the first
try.

The headline: long agentic sessions stay warm

If you drive MTPLX from OpenCode, Claude Code, Pi, or any agent that resends
its transcript every turn, this is the release to take.

• The session cache ceiling near 38k tokens is gone (#255, #269). The
  post-turn commit estimated an oversized snapshot and skipped both the write
  and the frontier update, so the committed frontier froze around 38k tokens
  and every later turn re-prefilled the whole transcript. Sessions past that
  point paid minutes of prefill for a turn that should have taken seconds.
  The oversized case now takes a zero-byte live-reference lease at the full
  frontier, so the next turn prefills only what is actually new. Thanks to
  @kmike for the receipts that pinned the plateau at exactly 38,335 tokens.
• Session commits byte-extend again (#269). The commit path built its
  banked prefix without the committed-reasoning substitution the next request
  actually sends, so commits failed every turn with
  retokenized_prefix_not_extending_session and cache reuse sat at 3 to 4
  percent. Commit and gate now share one canonicalization choke point, proven
  end to end with the real Qwen 3.8 tokenizer.
• The canonicalization gate cannot inject the wrong reasoning. It refuses
  on tool-call changes and dropped turns instead of substituting by position,
  handles OpenCode's stripped preambles, and repair re-encodes preserve
  committed reasoning. Two system contracts moved from prefix to suffix so
  flipping them no longer re-prefills the whole context.
• AR mode joins the session bank (#246). --no-mtp runs now restore warm
  prefixes and report real cache stats, so a speculative-versus-plain control
  arm measures decode alone. Before this, the AR arm silently paid a full
  re-prefill every request and its numbers were unusable as a control.

For anyone benchmarking MTPLX

We want MTPLX measured, so 2.8.0 makes the measurement surface honest and
hard to misread.

• Prompt scoring for KL quality harnesses. /v1/completions with
  echo: true, logprobs, and max_tokens: 0 scores a whole prompt in one
  call. The arrays follow OpenAI echo semantics exactly: every array has
  length n, token_logprobs[0] is null, the scored token always appears in
  its own top-K map with its true value (string collisions included), and a
  token_ids array gives stable identity when display strings collide.
  top_logprobs[0] is an empty dict rather than null because several public
  harness parsers iterate entries and crash on null.
• Over-context requests fail loudly. A prompt that cannot fit returns a
  clear 400 with context_length_exceeded instead of silently generating one
  token, and a fitting prompt whose max_tokens exceeds the remainder is
  clamped with the clamp visible in stats. No more phantom rows at the long
  end of a context ladder.
• finish_reason is truthful everywhere. A length cap beats tool_calls
  in non-streaming chat, /v1/messages maps max_tokens before
  stop_sequence before tool_use, the completions stream trims stop
  strings identically to non-stream, and a capped thinking row either
  recovers its content or says why it is empty
  (content_empty_reason: truncated_inside_reasoning).
• Streamed text equals non-streamed text. The stream leaked the blank
  line the model emits after its thinking block as a content delta, so
  diffing the two transports at temperature 0 always mismatched. Streamed
  content now concatenates to exactly the non-stream text.
• AR responses report honest numbers. No fabricated draft temperature on
  any lane, batched AR included, and the post-response bookkeeping forward
  pass is no longer billed into measured time on either the AR or the MTP
  lane.
• Repetition-guard stops stay off the wire. All streaming lanes hold back
  a detector-window tail while the guard is armed, so trimmed loop output
  never reaches a client, and a triggered stop is visible in public stats.
• /health reports degradation. A new degradation block says when
  compiled verify fell back to eager and why, which profile env keys an
  operator override beat, and the kernel bail counters. mtplx doctor prints
  the compiled-verify fence, including in --summary. "Looks like turbo,
  runs slow" is no longer invisible (#255).
• Richer per-response stats, stamped only when they apply:
  finish_reason, draft-sampler policy and ownership, greedy coupling,
  repetition-stop, content-empty reason, and clamp stats. The benchmarking
  guide documents response caps, thinking-off settings for capped harnesses,
  and the exact prompt-scoring contract.
• Bench entries stop paying hidden costs (#261). Metal memory caps and
  the over-context refusal apply to every bench, ladder, one-shot, and
  quickstart entry, rows flush as they complete, and compiled-verify prewarm
  happens outside measured rows and warms the exact traces real rows use.
  Thanks to @ArthurOstapenko for the report.

First run and every run after

• Sharing the API over your network is one line.
  mtplx serve --host 0.0.0.0 --api-key-file ~/.mtplx/api-key creates the
  key file if it is missing (0600, printed once) instead of crashing on the
  exact command our own error message suggests, and startup prints a
  Network OpenAI API Base URL with your Mac's LAN address, which is what a
  Parallels or VM guest should use. Keyless non-localhost binds still refuse.
  We are not shipping an open LLM port; we are shipping a working path to a
  keyed one.
• Quickstart leads with Auto. The wizard's first choice is now
  "Auto (recommended)", which pins nothing and lets the engine resolve the
  fastest verified profile per model. A previously saved wizard default of
  sustained migrates to Auto once; deliberate picks stay pinned. The macOS
  app's Auto likewise stopped emitting a profile flag, so renamed and legacy
  model folders no longer launch pinned to the slow profile.
• No surface claims sustained for a flagship anymore. Forge stamps,
  model listings, doctor, bench suites, tune, and the quickstart download
  branch all report what serve actually resolves.
• Branded local builds resolve by id. A forge-built pack under its bare
  folder name no longer makes quickstart demand a 20 GB re-download of a
  model it already has.
• Discover shows every MTPLX build. Case-insensitive name matching, no
  more slicing the top 30 by downloads before filtering, and a 100-row page,
  so the Qwen 3.8 family and community builds actually appear.
• Config values are real pins. A profile or sampler value in
  config.toml is honored as explicit in both directions and startup prints
  one line saying where it came from.
• Model identity comes from the artifact (#268). Family resolves from
  forge provenance first, and the 3.8 marker is boundary-guarded so stock
  Qwen/Qwen3-8B cannot be claimed by it. The macOS app now uses the same
  guard and the same provenance-first order. Thanks to @mmmugh for the
  original report and the marker analysis.
• KV quantization actually saves memory. The q8 mirror is offset-sized
  and released once the kernel path engages, q4 never allocates one,
  numerics route once per request, and the CLI text states the honest
  contract. Paged q8 decode runs through a dedicated kernel with counters in
  stats.
• The launch environment cannot steer requests. MTPLX_CLIENT is an
  observability label; client-specific behavior requires per-request
  evidence. An anonymous benchmarker's settings are honored as sent, even
  against an app-launched daemon. Claude Code's user agent is now recognized
  for observability.
• Streams end honestly. The wait between last content and the finish
  frame is bounded with live heartbeats and a watchdog, explicit cancels
  emit a terminal frame and [DONE], and client disconnects are tagged as
  disconnects.
• /v1/messages conformance. Parallel tool use, streamed usage
  accounting, and strict rejection of previously ignored fields.
• reasoning_effort: "high" maps up the engine's real ladder instead of
  silently using the default; unknown values return 400.
• The web chat UI names whose context cap it shows. A memory-capped
  launch reads "this server's 16.4k context window", not "the model's".
• Faster streaming under load. The SSE hot path uses a loop-fed queue
  and a constant envelope, cutting per-token server overhead at high decode
  speeds.

Still open

• With reasoning off, in a plain chat with no tools, Qwen 3.8 can still emit
  a stray tool call and end the turn early. Leave thinking on. Unchanged from
  2.7.1.
• A stream that ends in a tool call can still carry a trailing blank line
  ahead of the tool call in some chunkings. Cosmetic; non-stream responses
  and plain-text streams are byte-exact.
• Dense 27B models serve concurrent requests through a serialized MTP queue
  by default. That is a deliberate trade: serialized MTP wins prefill-heavy
  agentic ...

MTPLX 2.7.2

MTPLX 2.7.2

An emergency fix for mtplx pull. On 2.7.1 and older, re-pulling a model
that changed upstream — such as the Qwen 3.8 repos, re-published on
2026-08-15 with their vision towers restored — can corrupt your local copy.
Upgrade before you pull.

Fixes

• mtplx pull no longer corrupts files that changed upstream (#258,
  #234). The downloader treated a complete local file whose size no
  longer matched the server as an interrupted download, and byte-range
  appended the remote tail onto the old content. Updating a repo in place
  corrupted config.json and model.safetensors.index.json and left the
  model unloadable. Stale files are now discarded and re-fetched whole;
  genuinely interrupted *.incomplete downloads still resume. (Only pulls
  with progress reporting — the interactive CLI, --progress-json, and the
  app — had the bug; mtplx pull --json routes through hf_hub's etag
  downloader and was never affected.)

  Already hit by this? A file the old downloader corrupted ends up at
  exactly the size the server reports, so mtplx pull on 2.7.2 still sees
  it as complete and cannot repair it on its own. Delete the affected
  model's config.json and model.safetensors.index.json from its folder
  under ~/.mtplx/models/, then run mtplx pull again on 2.7.2.

• The Qwen 3.8 models can see again (#263). All six published 3.8 repos
  (Bare Speed, Optimized Speed, Optimized Quality, and their FP16 siblings)
  shipped without their vision towers: the forge convert lane kept only the
  text model. The repos were re-published on 2026-08-15 with the official
  bf16 tower grafted back in as an index-registered
  model-vision.safetensors; language and MTP tensors are untouched, so an
  existing install picks the repair up as a ~0.9 GB delta. mtplx forge build now grafts the vision tower, vision_config, and preprocessor
  sidecars on every lane and fails closed rather than producing a blind
  artifact (a repair script for already-forged artifacts ships as
  scripts/graft_vision_tower.py). Catalog and app download sizes now
  include the tower.

Still open

• With reasoning off, in a plain chat with no tools, Qwen 3.8 can still emit
  a stray tool call and end the turn early. Leave thinking on. (Unchanged
  from 2.7.1.)

Upgrading

• Upgrade before pulling the repaired Qwen 3.8 repos.
• CLI: pip install -U mtplx or brew upgrade mtplx.
• App: Sparkle will offer 2.7.2 (build 2007002), or grab the DMG.
• If a pull on 2.7.1 already corrupted a model, see the first fix above for
  the two files to delete before pulling again.

MTPLX 2.7.1

MTPLX 2.7.1

A bug-fix release. It clears the known-issues list 2.7.0 shipped with.

Fixes

• xhigh stays selected. Picking it in Inference settings while the model
  was running snapped straight back to medium, and
  mtplx config set reasoning_effort xhigh was refused outright. Those two
  places each carried their own copy of the effort list and neither had been
  told about xhigh, even though the engine had understood it since 2.7.0.
  The live-settings save is all-or-nothing, so one unrecognized level threw
  away the entire save and the picker reverted. Every place that accepts an
  effort level now reads the same list, and which levels a given model offers
  is still decided by that model alone.
• KV cache quantization reaches Qwen 3.8. The toggle displayed q8 but
  the launch path only recognized Qwen 3.5 and 3.6, so a 3.8 run quietly got
  nothing.
• mtplx doctor names the model it actually checked rather than the old
  default, and turbo's profile note reports the real compiled-verify fence of
  32,768 instead of the pre-2.7.0 number.
• A new build can't offer you an older one. Updates are ordered by build
  number, and the one derived for 2.7.1 came out below the 2.7.0 already in
  the wild — so a fresh 2.7.1 proposed 2.7.0 to itself. Fixed at the
  derivation, and the update feed now reads its number off the built app.

Still open

• With reasoning off, in a plain chat with no tools, Qwen 3.8 can still emit
  a stray tool call and end the turn early. Leave thinking on.

Upgrading

• App: Sparkle will offer 2.7.1 (build 2007001), or grab the DMG.
• CLI: pip install -U mtplx or brew upgrade mtplx.

MTPLX 2.7.0

Qwen3.8 support 🎉

Qwen3.8-27B came out on 14 August. This release runs it the way the model
card says it should be run, with three MTPLX builds tuned for it, FP16
versions of all three for M1 and M2 Macs, and a longer compiled verify
window that helps every model. It also fixes a CPU drain in the SSD session
cache and a crash on macOS 27.

Every speed number below was measured on one M5 Max with fans verified at
maximum, die temperature gated before each run, one request at a time,
generation running to the model's own stop. Other Macs will land elsewhere.
Nothing here was measured on M1 or M2.

Qwen 3.8, served properly

There is a new qwen3_8 model family in the engine and the app. It carries
Qwen's official inference contract instead of the Qwen 3.6 coding defaults:

• Sampling at temperature 1.0, top-p 0.95, top-k 20.
• Reasoning effort levels xhigh, medium and low. Coding sessions
  default to medium: on the same uncapped agent task, medium finished
  correct in 51.5 s where xhigh took 314.9 s. You can pick xhigh per
  request, on the CLI (--reasoning-effort xhigh) or in the app before you
  start the model.
• Thinking is preserved in the conversation history by default, which is
  what the model was trained on. Reasoning tokens stay in context and flow
  through MTP drafting like any other token, so speculation is calibrated on
  the thinking phase and the answer phase both.
• Qwen's chat_template_kwargs: {"enable_thinking": ...} request field is
  honored, so client code copied from the model card works unchanged.
• The Qwen 3.8 MTP head is trained for deeper drafts, but depth 4 killed the
  daemon on drop day, so live serving is capped at depth 3 in this release
  and Tune offers AR to D3.

The 3.8 trunk keeps the 3.6 hybrid attention layout, so the whole kernel
stack transfers as is: compiled verify graphs, the custom verify kernels and
the GQA fast paths engage identically, with the same load-time self-check on
your own chip.

Three builds, calibration included

• Bare Speed (16.0 GB): flat 4-bit. Quickest burst chat speeds. Lower
  quality and slower on long coding tasks.
• Optimized Speed (20.4 GB): 4-bit dynamic quant. Great coding speeds
  and good quality. Recommended. Same hand-tuned layout as Qwen 3.6
  Optimized Speed V2 (embeddings, output head, all 48 GDN output projections
  and the last 8 MLP blocks at 8-bit; GDN convolution and recurrent-state
  parameters, every norm and the whole MTP head at 16-bit).
• Optimized Quality (29.4 GB): 8-bit dynamic quant. Good coding speeds
  and perfect quality. Closest to the official bf16 model: KL divergence to
  the bf16 teacher 0.00105, against 0.0220 for Optimized Speed and 0.0376
  for Bare Speed.

Each build states its measured calibration in its own runtime metadata:
recommended draft sampler (Bare Speed 0.6, the Optimized pair 1.0), tuned
MTP depth (3 for all three), and peak memory measured on that build. The
runtime reads that metadata ahead of profile fallbacks, so a build launches
at its own tuned depth even when the serving profile disagrees, and the app
and the CLI launch every 3.8 build identically because neither pins a draft
sampler of its own any more.

Sizes shown in the app and CLI are the exact byte sums of the published Hub
files, and the peak memory numbers are measured, not inherited from a 3.6
sibling.

What the default is now

• Modern Apple Silicon (M3, M4, M5) with 32 GB or more: Qwen 3.8 Optimized
  Speed, downloaded on first use.
• M1 and M2 with 32 GB or more: Qwen 3.8 Optimized Speed FP16 (below).
• Under 32 GB: still the Qwen 3.5 9B route.

mtplx quickstart, mtplx start and the app's first-run picker offer the
whole 3.8 line-up in the same order with the same one-line descriptions:
Optimized Speed as the recommended default, then Bare Speed, then Optimized
Quality (Quality is hidden on 32 GB Macs because its measured 33 GB peak
does not fit there). Qwen 3.6 Optimized Speed V2 stays directly below them
and keeps its turbo profile.

If you already use MTPLX and your last run used the recommended default,
mtplx start says once that the default moved and from which model, instead
of quietly relabeling. The app keeps whatever model you had; it does not
switch you.

FP16 builds for M1 and M2

M1 and M2 have no native bf16, so every 3.8 build has an FP16 sibling on the
Hub (Youssofal/Qwen3.8-27B-MTPLX-...-FP16). They are the identical model:
every quantized pack is byte for byte the parent's (498 of 498 per build),
and every 16-bit tensor is the bf16 value cast to fp16 (99.992% of elements
exact; the remaining 0.008% are magnitudes below 7.6e-6 rounded on the fp16
subnormal grid, largest error 3.0e-8, none overflow). No bf16 tensor is left
in any of them.

The M1/M2 tier of the CLI and the app routes to the FP16 siblings
automatically: same three picks, same order, same descriptions, and the
OpenCode config names the id the server actually advertises
(mtplx-qwen38-27b-...-fp16). All three launch on turbo like their parents
and pass the fp16 kernel self-check at boot; if a chip ever disagrees, the
affected lane falls back to the stock path on its own.

Compiled verify to 32k

Since July the compiled verify graph stopped at 12,288 tokens of context,
because a KV copy tax at the time made longer compiled windows a regression.
That tax is gone, so turbo now compiles verify to 32,768 tokens. Interleaved
A/B on Qwen 3.8 Bare Speed under die-temperature gates: the compiled path
beat the eager fallback in every paired epoch (48.5 against 45.4 tok/s at 20k
context, +6.9%), with flat peak memory at 20k and lower at 30k (25.4 GB
against 28.5 GB; the eager path is the one that spikes). Past the fence the
same custom kernels run eagerly, exactly as before. MTPLX_COMPILED_VERIFY
can now be set by hand for parity and exactness runs against the shipped
profile.

Coding agents

• OpenCode and Pi no longer send an output cap of any kind. OpenCode injected
  a 32k ceiling even when the model advertised more; Pi silently substituted
  16,384 when the metadata omitted it. Both integrations now clear the
  generated cap for MTPLX models only, so generation runs to the model's own
  stop. Explicit caps you set yourself still apply.
• Pi sessions carry their real session id to MTPLX, so multi-turn Pi work
  restores its banked prefix from RAM instead of re-prefilling. Live receipt:
  16.9k to 18.5k tokens restored per turn across a five-turn coding task.
• The session bank's background re-render now uses the effort the request
  actually ran with, in the postcommit path and the idle scheduler lane.
  Before this, a medium-effort session could run the bank permanently cold
  while xhigh warm-hit, because the effort instruction is part of the
  rendered prompt.
• Reasoning that hits the token limit before the closing think tag is now
  routed as reasoning, not shown as the answer.

App

• Qwen 3.8 launch family: turbo by default, the official sampler preset,
  the reasoning effort control with xhigh available and medium as the
  coding default, Tune from AR to D3, and catalog rows with exact sizes and
  measured peaks.
• The first-run picker shows the 3.8 trio (FP16 siblings on M1 and M2), and
  the Qwen 3.6 Optimized Quality row on M1 and M2 now resolves to its FP16
  build instead of the bf16 one.
• Fixed a crash on macOS 27 when the inference settings overlay opened
  (#256, #257). SwiftUI 8 traps on a slider whose range has no distinct
  values; the depth slider was built with 1...1 for models without draft
  control, and the context-window slider could hit 4096...4096. Both are
  now built only when there is something to slide. Reported and fixed by
  @joshlacal.

Fixes

• SSD session cache no longer walks its whole store on every write or every
  /health poll. On a long-lived bank (816,220 files, 89.9 GB) each walk
  took 41.7 s. The cap check forced one per write, and the app's health
  poller kept another running back to back: most of a CPU core, all the
  time, heating the die under live decode. Reconciliation is now
  maintenance. It runs only when the store changed and at most 5% of the
  time, off the writer lock, yielding to live traffic, and the cap gate
  prices orphan bytes from the last snapshot instead of walking again.
  Measured on that bank: idle CPU with a health poller 35% down to 0.2%,
  per-write cap gate 71 to 159 s down to 3 to 6 s, cache-hit restores
  unchanged.
• mtplx pull names the mirror knob when a download fails for a network
  reason and no HF_ENDPOINT is set (#259): HF_ENDPOINT=https://hf-mirror.com
  on the CLI, Settings, Advanced, HF download mirror in the app. Both were
  already supported and neither was documented; the troubleshooting docs now
  cover them.
• mtplx doctor judges memory against the model this Mac would actually
  default to (9B under 32 GB, FP16 on M1/M2) instead of pricing a 27B
  against 80% of RAM and failing every Mac under about 45 GB. M5 Max is
  listed in the support matrix.
• Hardware detection calls /usr/sbin/sysctl and /usr/sbin/system_profiler
  by absolute path, so it works from the app's sanitized environment on any
  Mac. mtplx doctor and mtplx tune no longer run git outside a
  repository, which on a Mac without the Command Line Tools used to pop the
  Xcode install dialog during onboarding.
• Depth-default resolution honors artifact metadata across profile
  mismatches; the degrade pin (AR mode when the MTP head is missing) and the
  no-metadata path both survive. An early Quality build ran at depth 2
  instead of its tuned depth 3 because of this; the fix is worth +19% on
  that build.
• The public depth ceiling is decided by the artifact reference, not the
  served-name alias, so a non-3.8 model served under the default id cannot
  widen its own depth gate. mtplx tune validates depths against what the
  model actually supports and takes its sampler from the same family
  cont...

MTPLX 2.6.0

MTPLX 2.6.0 — concurrency

Until now, speculative decoding was a single-user feature: the moment two
requests hit the daemon at once, everyone fell back to plain autoregressive
batching and lost the MTP speedup. 2.6.0 removes that trade-off. This release
also brings embedding and reranking endpoints, LiquidAI LFM2 support, and a
real correctness fix to temperature-0 decoding.

Concurrent speculative decoding (--scheduler-mode mtp_batch)

The new scheduler serves independent requests through fixed-width MTP
cohorts. Each row owns its own state and sampling decisions, drafts are
verified in one batched target forward, and rows join and leave mid-flight
without disturbing their neighbours. Two cohort widths (three-wide and
eight-wide) install side by side and the scheduler seals each cohort at the
narrowest width that fits, so two concurrent agents don't pay for eight
lanes of padding.

Measured on Qwen3.6-35B-A3B on an M5 Max: the three-wide lane holds
1.7-1.8x the per-request decode of the padded eight-lane shape, and against
the previous production ar_batch route the same concurrent agent workloads
decode at 1.6-2.25x per lane, sampled at the model's shipped settings.

Honesty controls ship with it: --mtp-batch-numerics picks between
throughput, balanced, and b1-exact profiles with documented
trade-offs and an install-time self-check, and per-request stats report
each row's own truth (its own accepted-depth histogram, cohort width, and
restore provenance) instead of cohort averages.

The session bank composes with the cohorts: a request whose prefix is
banked restores it at cohort admission, prefills only its uncovered suffix,
and commits its own prompt boundary before the merge — agent fleets with a
shared system prompt keep warm time-to-first-token under concurrency. The
plain ar_batch lane learned the same trick.

This work is by David Tai (@davidtai): the scheduler contract, the
row-owned decode, the cohort serving, the numerics profiles, and the
docs. The width-3 bucket, session-bank composite, and live QA came out of
the joint hardening passes on top.

Embeddings and reranking (/v1/embeddings, /v1/rerank)

Contributed by @Cyb3rb1ade (PR #212). The daemon can now serve embedding
and reranker models beside chat, so a retrieval-backed setup doesn't need a
second inference server. OpenAI-shape embeddings (including the
dimensions Matryoshka truncation), Cohere/Jina-shape rerank, opt-in
per-model flags, lazy loading, an LRU resident cap, and idle release under
memory pressure. /v1/models stays chat-only by default so model pickers
never offer an embedder as a chat target; retrieval-only ids answer chat
requests with a clear 400. Checkpoints that ship their own Python code are
refused unless you explicitly pass --retrieval-trust-remote-code.

LiquidAI LFM2 / LFM2.5

By David Tai (@davidtai). The LFM2 family serves natively with a bit-exact
ShortConv decode fast-path and a verified think/tool grammar (parser stamp,
native tool prompt, pythonic streaming dialect). IQuest-Coder checkpoints
serve target-only AR through the same registry honesty: recognized, served
without MTP claims, refused cleanly when the quantization can't execute.

Temperature-0 output is token-identical again

The speculative lane's cold prefill fed the whole prompt through the model
in one window while plain decoding splits it into body plus a final
single-token step. The two shapes round differently in the last bit, so the
speculative lane started from a cache one ulp apart from the plain lane's —
enough to flip greedy argmax at a near-tie and break the "temperature 0
matches plain decoding" contract. Every cold-prefill path now partitions
the prompt identically. The Optimized Speed V2 artifact, which surfaced the
flip, passes its greedy exactness gate at every depth again.

Fixes

• Prefix restores no longer corrupt the session bank (#247): restores
  install fresh zero-copy views, so an interleaved near-prefix request can
  never rewrite a neighbour's banked span. From the reporter's reproducer,
  which ships as a regression test.
• Streaming tool calls no longer duplicate argument values into
  delta.content (#249).
• Solo requests carrying presence/frequency penalties answer on the
  composite scheduler instead of returning HTTP 500.
• mtplx serve --no-auth actually parses (#235 follow-through) — 2.5.4
  promised it, the public CLI rejected it.
• qwen3_5_mtp checkpoints validate and serve again — the MTP surface now
  attaches at the TextModel level. Cherry-picked from PR #242 by
  @davidtai with its regression test.
• Artifacts launch at their measured depth again: the typed runtime
  contract silently dropped measured-depth maps, so 35B launched at its D3
  ceiling — a measured ~22% decode loss against its fastest depth.
• --reasoning-parser is authoritative; the backend codec no longer
  silently overrides an operator-typed parser.
• Metal buffer-object leak in long decodes: mlx-lm's ArraysCache regrew
  buffer objects on every advance; the fix is vendored in-tree so pip
  installs against stock mlx-lm 0.31.x get it too.
• Missing MTP heads degrade to target-only AR serving with the reason
  surfaced, instead of refusing the checkpoint. auto_map checkpoints are
  refused with the policy stated plainly; unrunnable quantizations are
  refused with the offending bit-width named.
• OpenAI dimensions honored on /v1/embeddings; out-of-range values are a
  clear 400.
• AR batch hardening: cache-removal errors fail closed, completed streams
  no longer starve behind running neighbours, and the vendored Metal shader
  cache is keyed by MLX ABI so an MLX upgrade can't serve stale kernels.

QA (this release)

• Full pytest battery green at the release tip; 567 Swift app tests green.
• Greedy exactness gate for Optimized Speed V2: 3/3 depths token-exact
  (was 0/3 on 2.5.4).
• Four-arm performance sweep against the shipped 2.5.2, 2.5.3, and 2.5.4
  wheels (same harness bytes, fans verified at max, die-temp gated,
  candidate interleaved with baseline): single-request decode
  flat-to-faster than 2.5.4 (interleaved means 74.8 vs 69.2 tok/s on the
  27B artifact), cold TTFT flat (6.25s vs 6.26s on a 4.8k-token prompt),
  warm TTFT flat at the 2.5.4 session-bank floor (0.14s vs 0.15s). A
  serial-lane sampling regression introduced mid-cycle was caught by this
  sweep and fixed before release.
• Live QA on both product surfaces at the release tip: mtplx serve from
  a clean wheel install (streamed think and content, 64-70 tok/s) and the
  macOS app driven end-to-end (chat round trip, 52.5 tok/s reported by
  the app, clean stop and quit).

Credits

• David Tai (@davidtai) — concurrent MTP serving stack, LFM2 support,
  qwen3_5_mtp fix (PR #242). 26 commits in this release.
• @Cyb3rb1ade — embeddings + rerank endpoints (PR #212, 17 commits).