v0.0.2

v0.0.2 — speed + lossless sampling + an honest performance model

Faster drafting, a new sampling mode, and measured guidance on what to expect on Apple Silicon. Output stays lossless: greedy is byte-identical to plain greedy decoding, and --temperature > 0 is an exact sample from the target at temperature T.

Highlights

• Lossless temperature speculative sampling (--temperature, --seed) — the paper's actual method (§2.1): sample the draft, accept with prob min(1, p/q), resample the residual on rejection. Validated that the output distribution exactly equals the target's. (Default stays greedy / temperature 0.)
• ~9–10% faster at the default cap from a drafter-slice fix: the block's lm_head + Markov head now run only over the verified positions instead of the full 7 every round. Output-neutral.
• --max-draft 2 is the new default — the measured optimum on Apple Silicon (the per-token verify cost makes longer blocks slower for typical acceptance). Was 4.
• 4-bit target option (--target mlx-community/gemma-4-12B-it-4bit) — highest absolute throughput and fits ≤24 GB Macs (at some quality cost). 8-bit remains the default sweet spot.

Changed

• Default max_draft_tokens 4 → 2.
• Confidence-based truncation now uses the cumulative prefix-survival product ∏ c_i (paper Eq 7-8) instead of a per-position threshold. Kept as an option; not the default.
• greedy_generate gains temperature / seed (so --mode baseline can sample for fair A/Bs).

Performance (M4 Pro, warm, vs the official mlx_lm / mlx_vlm tools)

model	baseline	mlx-dspark	speedup
Gemma-4 12B	18.4 tok/s (mlx_vlm)	~30 tok/s	~1.6× (up to ~2× on code/math)
Qwen3-4B	52.9 tok/s (mlx_lm)	~73 tok/s	~1.4×

This matches DSpark's own per-user serving claim (~1.57–1.85×). The README documents the
Apple-Silicon cost model / ceiling and an on-device DSpark-vs-DFlash comparison.

Notes

• Greedy decoding path is unchanged and remains greedy-correct.
• Drafter quantization does not affect acceptance (4/8-bit/bf16 identical); 4-bit stays default.
• No API breakage; new arguments are optional with greedy-preserving defaults.

Full diff: v0.0.1...v0.0.2