MTPLX 2.7.0

Qwen3.8 support 🎉

Qwen3.8-27B came out on 14 August. This release runs it the way the model
card says it should be run, with three MTPLX builds tuned for it, FP16
versions of all three for M1 and M2 Macs, and a longer compiled verify
window that helps every model. It also fixes a CPU drain in the SSD session
cache and a crash on macOS 27.

Every speed number below was measured on one M5 Max with fans verified at
maximum, die temperature gated before each run, one request at a time,
generation running to the model's own stop. Other Macs will land elsewhere.
Nothing here was measured on M1 or M2.

Qwen 3.8, served properly

There is a new qwen3_8 model family in the engine and the app. It carries
Qwen's official inference contract instead of the Qwen 3.6 coding defaults:

• Sampling at temperature 1.0, top-p 0.95, top-k 20.
• Reasoning effort levels xhigh, medium and low. Coding sessions
  default to medium: on the same uncapped agent task, medium finished
  correct in 51.5 s where xhigh took 314.9 s. You can pick xhigh per
  request, on the CLI (--reasoning-effort xhigh) or in the app before you
  start the model.
• Thinking is preserved in the conversation history by default, which is
  what the model was trained on. Reasoning tokens stay in context and flow
  through MTP drafting like any other token, so speculation is calibrated on
  the thinking phase and the answer phase both.
• Qwen's chat_template_kwargs: {"enable_thinking": ...} request field is
  honored, so client code copied from the model card works unchanged.
• The Qwen 3.8 MTP head is trained for deeper drafts, but depth 4 killed the
  daemon on drop day, so live serving is capped at depth 3 in this release
  and Tune offers AR to D3.

The 3.8 trunk keeps the 3.6 hybrid attention layout, so the whole kernel
stack transfers as is: compiled verify graphs, the custom verify kernels and
the GQA fast paths engage identically, with the same load-time self-check on
your own chip.

Three builds, calibration included

• Bare Speed (16.0 GB): flat 4-bit. Quickest burst chat speeds. Lower
  quality and slower on long coding tasks.
• Optimized Speed (20.4 GB): 4-bit dynamic quant. Great coding speeds
  and good quality. Recommended. Same hand-tuned layout as Qwen 3.6
  Optimized Speed V2 (embeddings, output head, all 48 GDN output projections
  and the last 8 MLP blocks at 8-bit; GDN convolution and recurrent-state
  parameters, every norm and the whole MTP head at 16-bit).
• Optimized Quality (29.4 GB): 8-bit dynamic quant. Good coding speeds
  and perfect quality. Closest to the official bf16 model: KL divergence to
  the bf16 teacher 0.00105, against 0.0220 for Optimized Speed and 0.0376
  for Bare Speed.

Each build states its measured calibration in its own runtime metadata:
recommended draft sampler (Bare Speed 0.6, the Optimized pair 1.0), tuned
MTP depth (3 for all three), and peak memory measured on that build. The
runtime reads that metadata ahead of profile fallbacks, so a build launches
at its own tuned depth even when the serving profile disagrees, and the app
and the CLI launch every 3.8 build identically because neither pins a draft
sampler of its own any more.

Sizes shown in the app and CLI are the exact byte sums of the published Hub
files, and the peak memory numbers are measured, not inherited from a 3.6
sibling.

What the default is now

• Modern Apple Silicon (M3, M4, M5) with 32 GB or more: Qwen 3.8 Optimized
  Speed, downloaded on first use.
• M1 and M2 with 32 GB or more: Qwen 3.8 Optimized Speed FP16 (below).
• Under 32 GB: still the Qwen 3.5 9B route.

mtplx quickstart, mtplx start and the app's first-run picker offer the
whole 3.8 line-up in the same order with the same one-line descriptions:
Optimized Speed as the recommended default, then Bare Speed, then Optimized
Quality (Quality is hidden on 32 GB Macs because its measured 33 GB peak
does not fit there). Qwen 3.6 Optimized Speed V2 stays directly below them
and keeps its turbo profile.

If you already use MTPLX and your last run used the recommended default,
mtplx start says once that the default moved and from which model, instead
of quietly relabeling. The app keeps whatever model you had; it does not
switch you.

FP16 builds for M1 and M2

M1 and M2 have no native bf16, so every 3.8 build has an FP16 sibling on the
Hub (Youssofal/Qwen3.8-27B-MTPLX-...-FP16). They are the identical model:
every quantized pack is byte for byte the parent's (498 of 498 per build),
and every 16-bit tensor is the bf16 value cast to fp16 (99.992% of elements
exact; the remaining 0.008% are magnitudes below 7.6e-6 rounded on the fp16
subnormal grid, largest error 3.0e-8, none overflow). No bf16 tensor is left
in any of them.

The M1/M2 tier of the CLI and the app routes to the FP16 siblings
automatically: same three picks, same order, same descriptions, and the
OpenCode config names the id the server actually advertises
(mtplx-qwen38-27b-...-fp16). All three launch on turbo like their parents
and pass the fp16 kernel self-check at boot; if a chip ever disagrees, the
affected lane falls back to the stock path on its own.

Compiled verify to 32k

Since July the compiled verify graph stopped at 12,288 tokens of context,
because a KV copy tax at the time made longer compiled windows a regression.
That tax is gone, so turbo now compiles verify to 32,768 tokens. Interleaved
A/B on Qwen 3.8 Bare Speed under die-temperature gates: the compiled path
beat the eager fallback in every paired epoch (48.5 against 45.4 tok/s at 20k
context, +6.9%), with flat peak memory at 20k and lower at 30k (25.4 GB
against 28.5 GB; the eager path is the one that spikes). Past the fence the
same custom kernels run eagerly, exactly as before. MTPLX_COMPILED_VERIFY
can now be set by hand for parity and exactness runs against the shipped
profile.

Coding agents

• OpenCode and Pi no longer send an output cap of any kind. OpenCode injected
  a 32k ceiling even when the model advertised more; Pi silently substituted
  16,384 when the metadata omitted it. Both integrations now clear the
  generated cap for MTPLX models only, so generation runs to the model's own
  stop. Explicit caps you set yourself still apply.
• Pi sessions carry their real session id to MTPLX, so multi-turn Pi work
  restores its banked prefix from RAM instead of re-prefilling. Live receipt:
  16.9k to 18.5k tokens restored per turn across a five-turn coding task.
• The session bank's background re-render now uses the effort the request
  actually ran with, in the postcommit path and the idle scheduler lane.
  Before this, a medium-effort session could run the bank permanently cold
  while xhigh warm-hit, because the effort instruction is part of the
  rendered prompt.
• Reasoning that hits the token limit before the closing think tag is now
  routed as reasoning, not shown as the answer.

App

• Qwen 3.8 launch family: turbo by default, the official sampler preset,
  the reasoning effort control with xhigh available and medium as the
  coding default, Tune from AR to D3, and catalog rows with exact sizes and
  measured peaks.
• The first-run picker shows the 3.8 trio (FP16 siblings on M1 and M2), and
  the Qwen 3.6 Optimized Quality row on M1 and M2 now resolves to its FP16
  build instead of the bf16 one.
• Fixed a crash on macOS 27 when the inference settings overlay opened
  (#256, #257). SwiftUI 8 traps on a slider whose range has no distinct
  values; the depth slider was built with 1...1 for models without draft
  control, and the context-window slider could hit 4096...4096. Both are
  now built only when there is something to slide. Reported and fixed by
  @joshlacal.

Fixes

• SSD session cache no longer walks its whole store on every write or every
  /health poll. On a long-lived bank (816,220 files, 89.9 GB) each walk
  took 41.7 s. The cap check forced one per write, and the app's health
  poller kept another running back to back: most of a CPU core, all the
  time, heating the die under live decode. Reconciliation is now
  maintenance. It runs only when the store changed and at most 5% of the
  time, off the writer lock, yielding to live traffic, and the cap gate
  prices orphan bytes from the last snapshot instead of walking again.
  Measured on that bank: idle CPU with a health poller 35% down to 0.2%,
  per-write cap gate 71 to 159 s down to 3 to 6 s, cache-hit restores
  unchanged.
• mtplx pull names the mirror knob when a download fails for a network
  reason and no HF_ENDPOINT is set (#259): HF_ENDPOINT=https://hf-mirror.com
  on the CLI, Settings, Advanced, HF download mirror in the app. Both were
  already supported and neither was documented; the troubleshooting docs now
  cover them.
• mtplx doctor judges memory against the model this Mac would actually
  default to (9B under 32 GB, FP16 on M1/M2) instead of pricing a 27B
  against 80% of RAM and failing every Mac under about 45 GB. M5 Max is
  listed in the support matrix.
• Hardware detection calls /usr/sbin/sysctl and /usr/sbin/system_profiler
  by absolute path, so it works from the app's sanitized environment on any
  Mac. mtplx doctor and mtplx tune no longer run git outside a
  repository, which on a Mac without the Command Line Tools used to pop the
  Xcode install dialog during onboarding.
• Depth-default resolution honors artifact metadata across profile
  mismatches; the degrade pin (AR mode when the MTP head is missing) and the
  no-metadata path both survive. An early Quality build ran at depth 2
  instead of its tuned depth 3 because of this; the fix is worth +19% on
  that build.
• The public depth ceiling is decided by the artifact reference, not the
  served-name alias, so a non-3.8 model served under the default id cannot
  widen its own depth gate. mtplx tune validates depths against what the
  model actually supports and takes its sampler from the same family
  contract as mtplx serve.
• MTPLX_REQUEST_LOG_JSONL=1 means "log to the default file", not a file
  literally named 1.
• First-live-contact serve fixes for 3.8: xhigh boot no longer trips strict
  warmup, and the request-log env toggle is honored on the family path.

Thermal honesty

• Every max-fan session now holds an ownership token. A daemon shutting down
  behind its replacement can no longer switch the fans back to Auto under
  the daemon that is still serving (or under your benchmark).
• Forge refuses to load a model for benchmarking when verified max-fan mode
  cannot start, instead of quietly reporting derated numbers. mtplx tune
  gained --require-max-fans for the same reason.

Forge

• Recipes can carry module_overrides: per-module quantization overrides
  applied in one conversion pass (suffix match, optional layer list, bits,
  group size, mode). This is the lane that built Optimized Speed.

Known issues, fixed in 2.7.1

• Choosing xhigh in the app's Inference settings while the model is
  already running is rejected by the server in this build, and
  mtplx config set reasoning_effort xhigh is rejected the same way. Set it
  before starting the model, or pass --reasoning-effort xhigh on the CLI.
• The app's KV cache quantization toggle is not applied to Qwen 3.8 models
  in this build.
• With reasoning switched off in a plain chat with no tools, Qwen 3.8
  emitted a stray tool call and cut the turn short on about half of our
  coding prompts. Keep thinking on (the default) until 2.7.1.

QA for this release

• Medium-effort coding instrument, identical prompt across engines: Bare
  Speed 65.2 tok/s, Optimized Speed 58.7 (accepted probability by depth
  0.961 / 0.879 / 0.816), Optimized Quality 40.6 (that run was taken before
  the depth fix, at depth 2; the installed-app line below has the shipped
  depth-3 number). Same instrument, same night, Qwen 3.6 Optimized Speed
  V2: 59.9 to 60.1 tok/s. The 3.8 Bare build outruns the 3.6 flagship.
• The installed app end to end (this release's signed bundle, engine
  started from the UI, defaults resolved purely from artifact metadata,
  cold sessions): Bare Speed 64.4 tok/s at 17.0 GB peak, Optimized Speed
  55.5 at 23.6 GB, Optimized Quality 48.3 at 32.7 GB. The speed the app
  shows matches the request-log receipt on every run.
• Head to head, same prompt and sampling: oMLX 0.5.7 serving its own
  Qwen3.8-27B 4-bit MTP quant with its speculative path on decoded 63.3
  tok/s. LM Studio on the long-form task: 17.40 tok/s against Bare Speed at
  32.4 tok/s sustained over a single 52,740-token response (27.2 minutes,
  ended at the model's own stop).
• xhigh long form: Optimized Speed 35.1 and 37.3 tok/s over 28k and 20k
  token responses; Optimized Quality 33.2 and 33.1 at depth 3; Bare Speed
  35.7 and 32.0 over 34k and 37k token responses.
• Verify cost per round on the medium instrument: Bare 44.0 ms, Optimized
  Speed 50.3 ms, against 51.5 and 52.4 ms for 3.6 V2 the same night.
• Live agent QA on ship defaults with no flags: a two-turn OpenCode coding
  session and a headless Pi session, both with warm session-bank restores
  from RAM at every turn, no re-prefill storms, no cache poisoning.
• Exactness: acceptance is the exact probability-ratio rule with residual
  resampling, so sampled output follows the target distribution at every
  temperature. Fixed-geometry determinism verified byte-identical on all
  three builds. At temperature 0, MTP-vs-AR argmax can still flip on
  near-ties across different verify tile geometries (the bf16 rounding
  property documented in #245), at the same rate as shipped 2.6.0.
• Release pipeline on the shipped commit: full Python battery green, Swift
  app suite 573 of 573, pillar QA (vision cache, memory ceiling, long-output
  decay) all pass with fans verified, app and DMG notarized and stapled.

Licensing

MTPLX stays Apache-2.0. The NOTICE file, which Apache-2.0 section 4(d) carries
with every copy, now requires attribution rather than preferring it: a product,
app, or service that includes or is built on MTPLX has to show "Powered by
MTPLX" and the project link inside the product, somewhere a user can see it
(About screen, credits, settings, shipped docs, or a CLI startup banner). A
mention in a repo or on a website does not cover it.

Thanks

@joshlacal for the macOS 27 crash report and fix (#256, #257), and everyone
who filed the download and mirror reports behind #259.