MTPLX 2.8.0

Use 2.8.1 instead.
The 2.8.0 wheel carried a vision session-cache defect for about an hour
after publish: a repeated transcript with a different image could restore
the previous image's cached context. Our release gate caught it before the
desktop build shipped; 2.8.1 is this release plus that fix.

MTPLX 2.8.0

This release is about trust. Over the last week people started benchmarking
MTPLX seriously and running long agentic sessions against it, and both groups
found real problems: sessions that quietly stopped reusing their cache past
38k tokens, stats that could disagree with what the engine actually did, and
a first-run path with sharp edges. 2.8.0 closes 45 commits of that work. The
rule for the whole release was simple: every number the server reports must be
the number the engine lived, and every documented path must work on the first
try.

The headline: long agentic sessions stay warm

If you drive MTPLX from OpenCode, Claude Code, Pi, or any agent that resends
its transcript every turn, this is the release to take.

• The session cache ceiling near 38k tokens is gone (#255, #269). The
  post-turn commit estimated an oversized snapshot and skipped both the write
  and the frontier update, so the committed frontier froze around 38k tokens
  and every later turn re-prefilled the whole transcript. Sessions past that
  point paid minutes of prefill for a turn that should have taken seconds.
  The oversized case now takes a zero-byte live-reference lease at the full
  frontier, so the next turn prefills only what is actually new. Thanks to
  @kmike for the receipts that pinned the plateau at exactly 38,335 tokens.
• Session commits byte-extend again (#269). The commit path built its
  banked prefix without the committed-reasoning substitution the next request
  actually sends, so commits failed every turn with
  retokenized_prefix_not_extending_session and cache reuse sat at 3 to 4
  percent. Commit and gate now share one canonicalization choke point, proven
  end to end with the real Qwen 3.8 tokenizer.
• The canonicalization gate cannot inject the wrong reasoning. It refuses
  on tool-call changes and dropped turns instead of substituting by position,
  handles OpenCode's stripped preambles, and repair re-encodes preserve
  committed reasoning. Two system contracts moved from prefix to suffix so
  flipping them no longer re-prefills the whole context.
• AR mode joins the session bank (#246). --no-mtp runs now restore warm
  prefixes and report real cache stats, so a speculative-versus-plain control
  arm measures decode alone. Before this, the AR arm silently paid a full
  re-prefill every request and its numbers were unusable as a control.

For anyone benchmarking MTPLX

We want MTPLX measured, so 2.8.0 makes the measurement surface honest and
hard to misread.

• Prompt scoring for KL quality harnesses. /v1/completions with
  echo: true, logprobs, and max_tokens: 0 scores a whole prompt in one
  call. The arrays follow OpenAI echo semantics exactly: every array has
  length n, token_logprobs[0] is null, the scored token always appears in
  its own top-K map with its true value (string collisions included), and a
  token_ids array gives stable identity when display strings collide.
  top_logprobs[0] is an empty dict rather than null because several public
  harness parsers iterate entries and crash on null.
• Over-context requests fail loudly. A prompt that cannot fit returns a
  clear 400 with context_length_exceeded instead of silently generating one
  token, and a fitting prompt whose max_tokens exceeds the remainder is
  clamped with the clamp visible in stats. No more phantom rows at the long
  end of a context ladder.
• finish_reason is truthful everywhere. A length cap beats tool_calls
  in non-streaming chat, /v1/messages maps max_tokens before
  stop_sequence before tool_use, the completions stream trims stop
  strings identically to non-stream, and a capped thinking row either
  recovers its content or says why it is empty
  (content_empty_reason: truncated_inside_reasoning).
• Streamed text equals non-streamed text. The stream leaked the blank
  line the model emits after its thinking block as a content delta, so
  diffing the two transports at temperature 0 always mismatched. Streamed
  content now concatenates to exactly the non-stream text.
• AR responses report honest numbers. No fabricated draft temperature on
  any lane, batched AR included, and the post-response bookkeeping forward
  pass is no longer billed into measured time on either the AR or the MTP
  lane.
• Repetition-guard stops stay off the wire. All streaming lanes hold back
  a detector-window tail while the guard is armed, so trimmed loop output
  never reaches a client, and a triggered stop is visible in public stats.
• /health reports degradation. A new degradation block says when
  compiled verify fell back to eager and why, which profile env keys an
  operator override beat, and the kernel bail counters. mtplx doctor prints
  the compiled-verify fence, including in --summary. "Looks like turbo,
  runs slow" is no longer invisible (#255).
• Richer per-response stats, stamped only when they apply:
  finish_reason, draft-sampler policy and ownership, greedy coupling,
  repetition-stop, content-empty reason, and clamp stats. The benchmarking
  guide documents response caps, thinking-off settings for capped harnesses,
  and the exact prompt-scoring contract.
• Bench entries stop paying hidden costs (#261). Metal memory caps and
  the over-context refusal apply to every bench, ladder, one-shot, and
  quickstart entry, rows flush as they complete, and compiled-verify prewarm
  happens outside measured rows and warms the exact traces real rows use.
  Thanks to @ArthurOstapenko for the report.

First run and every run after

• Sharing the API over your network is one line.
  mtplx serve --host 0.0.0.0 --api-key-file ~/.mtplx/api-key creates the
  key file if it is missing (0600, printed once) instead of crashing on the
  exact command our own error message suggests, and startup prints a
  Network OpenAI API Base URL with your Mac's LAN address, which is what a
  Parallels or VM guest should use. Keyless non-localhost binds still refuse.
  We are not shipping an open LLM port; we are shipping a working path to a
  keyed one.
• Quickstart leads with Auto. The wizard's first choice is now
  "Auto (recommended)", which pins nothing and lets the engine resolve the
  fastest verified profile per model. A previously saved wizard default of
  sustained migrates to Auto once; deliberate picks stay pinned. The macOS
  app's Auto likewise stopped emitting a profile flag, so renamed and legacy
  model folders no longer launch pinned to the slow profile.
• No surface claims sustained for a flagship anymore. Forge stamps,
  model listings, doctor, bench suites, tune, and the quickstart download
  branch all report what serve actually resolves.
• Branded local builds resolve by id. A forge-built pack under its bare
  folder name no longer makes quickstart demand a 20 GB re-download of a
  model it already has.
• Discover shows every MTPLX build. Case-insensitive name matching, no
  more slicing the top 30 by downloads before filtering, and a 100-row page,
  so the Qwen 3.8 family and community builds actually appear.
• Config values are real pins. A profile or sampler value in
  config.toml is honored as explicit in both directions and startup prints
  one line saying where it came from.
• Model identity comes from the artifact (#268). Family resolves from
  forge provenance first, and the 3.8 marker is boundary-guarded so stock
  Qwen/Qwen3-8B cannot be claimed by it. The macOS app now uses the same
  guard and the same provenance-first order. Thanks to @mmmugh for the
  original report and the marker analysis.
• KV quantization actually saves memory. The q8 mirror is offset-sized
  and released once the kernel path engages, q4 never allocates one,
  numerics route once per request, and the CLI text states the honest
  contract. Paged q8 decode runs through a dedicated kernel with counters in
  stats.
• The launch environment cannot steer requests. MTPLX_CLIENT is an
  observability label; client-specific behavior requires per-request
  evidence. An anonymous benchmarker's settings are honored as sent, even
  against an app-launched daemon. Claude Code's user agent is now recognized
  for observability.
• Streams end honestly. The wait between last content and the finish
  frame is bounded with live heartbeats and a watchdog, explicit cancels
  emit a terminal frame and [DONE], and client disconnects are tagged as
  disconnects.
• /v1/messages conformance. Parallel tool use, streamed usage
  accounting, and strict rejection of previously ignored fields.
• reasoning_effort: "high" maps up the engine's real ladder instead of
  silently using the default; unknown values return 400.
• The web chat UI names whose context cap it shows. A memory-capped
  launch reads "this server's 16.4k context window", not "the model's".
• Faster streaming under load. The SSE hot path uses a loop-fed queue
  and a constant envelope, cutting per-token server overhead at high decode
  speeds.

Still open

• With reasoning off, in a plain chat with no tools, Qwen 3.8 can still emit
  a stray tool call and end the turn early. Leave thinking on. Unchanged from
  2.7.1.
• A stream that ends in a tool call can still carry a trailing blank line
  ahead of the tool call in some chunkings. Cosmetic; non-stream responses
  and plain-text streams are byte-exact.
• Dense 27B models serve concurrent requests through a serialized MTP queue
  by default. That is a deliberate trade: serialized MTP wins prefill-heavy
  agentic loads end to end, batching wins short decode-heavy loads, and
  --scheduler-mode ar_batch opts into the latter. A speculative batched
  lane for dense models is on the roadmap.

Upgrading

• CLI: pip install -U mtplx or brew upgrade mtplx.
• App: Sparkle will offer 2.8.0 (build 2008000), or grab the DMG.
• No breaking API changes. Two response-shape notes for harness authors:
  top_logprobs[0] on echo scoring is now {} instead of null, and streamed
  chat content no longer begins with the blank line that followed the model's
  thinking block.