v0.3.2 — 453.70 tok/s: 24% past llama.cpp + trust-hardened eval pipeline
The lead over llama.cpp doubles to ~24%, and the evaluation that proves it is substantially hardened.
🏁 Performance — frontier 410.85 → 453.70 tok/s (+10.4%), now 24% past llama.cpp
Same RTX 5090, same Q4_K_M GGUF, warm & interleaved vs llama-bench:
| decode length | sparkinfer | llama.cpp | vs v0.3.1 | |
|---|---|---|---|---|
| 128 tok | 453.70 | 365.85 | +24.0% | was +12.1% |
| 256 tok | 443.53 | 364.90 | +21.6% | was +10.0% |
| 512 tok | 425.23 | 361.64 | +17.6% | was +6.7% |
The lead grew at every length — still a pure kernel win (same model/precision/greedy decode, no speculative decoding).
- #89 — Q/K/V projections on concurrent CUDA streams, byte-identical (@James-CUDA)
- #86 — single-pass MoE top-k (parallel rank-count) + fused RoPE/KV-append (@fansilas)
🔒 Trust-hardened evaluation pipeline (#102)
Closes the remaining gaming/poisoning vectors from the eval trust-model audit:
- Held-out prompts (H1) — a fresh unpredictable per-seed prompt each eval, so a PR can't overfit the gate; seed logged for reproduction.
- Reference quarantine (C2) — model weights (sha256-pinned) + llama.cpp (commit-pinned) verified/rebuilt each run; a tampered baseline can't skew a verdict.
- Clock record (M1) — graphics clock pinned where allowed and always recorded, so the absolute tok/s is reproducible.
- Immutable frontier ledger (H2) — every advance appends a GitHub-timestamped
(date, PR, author, commit, Δ%, prev→new)line; auditable line-by-line. - Provenance (clock, seed, reference pins) in every verdict + immutable log.
✅ KL metric fixed (honest, strict gate kept)
A held-out KL of 0.27 turned out to be a measurement artifact (shallow top-k dump floored llama's tail). The corrected matched-depth metric shows the true divergence is ~0.02 (top-1 0.97). Proven honest — a broken build reads KL 18; a 12-prompt sweep holds 0.007–0.022. So the strict KL ≤ 0.20 gate is kept, not loosened.
Verified: RTX 5090 453.70 tok/s, top-1 0.97 vs a fully-built CUDA llama.cpp.
Full notes: CHANGELOG.md.