v0.2.0
Speculative decode with the model's own draft head
The checkpoint ships a multi-token-prediction head that predicts the token
after next. The community conversion dropped those tensors; slotstream now
restores them with a one-time conversion from the official release (exactly
the 31 needed tensors via ranged downloads — 4.9 GB, ~2 minutes, not the full
checkpoint) and uses the head to draft a few tokens per step, verifying them
in one batched main-model pass.
- 85.8% measured first-draft accept rate; 3.5 accepted tokens per verify
round at depth 4 (measured across prose, code, lists, arithmetic). --mtp auto|on|offonrun/serve/doctor. Auto enables the head
only where the multiplier beats spending the same 1.6 GB on expert cache
(~120 experts/layer after the charge). Below that it measured ×0.96 — so
auto keeps it off there, and that A/B is in MEASUREMENTS.md.- The Swift port of the head is bit-exact against the MLX Python
reference (slotstream mtp-parity, a standing battery gate). - Sampling semantics are exact by construction: draws happen sequentially off
verified logits, only for tokens the plain loop would also sample — same
rng stream, same presence-penalty evolution, any temperature. - New gates:
mtp-check(determinism, cross-request state integrity through
the prefix cache, accept sanity) andmtp-accept(measure the accept curve
yourself).
Convert the head once (needs the pinned model already pulled):
.venv/bin/python Tools/mtp_convert.py
Without mtp.safetensors everything runs exactly as before, head off.