Repository navigation
v1.26.1 - the batching release
Install: pip install -U quantprobe (1.26.1 supersedes 1.26.0, whose wheel misreports its version string - functionally identical, see CHANGELOG.)
Since 1.25.0, all measured on the reference box and staked before measurement:
- C-06 overturned: the "2x batching ceiling" was our own sweep stopping at 8 streams. A GTX 1060 serves 32 concurrent 7B streams at 219 aggregate tok/s (9.5x single-stream); the jump sits exactly at llama.cpp's mat-vec width cap.
- U-39 confirmed as staked: batching does NOT survive expert offload (2.0x cap) - so the best model inverts with user count: 1 user -> 30B MoE; 32 users -> dense 7B, 219 vs 40 aggregate.
plannow prints both halves as placement-gated serving advice. - X-1: speculation's draft length is a kernel decision - drafts of 4-7 are strictly dominated on pre-Ampere (48-51 tok/s) vs m>=8 (88.5-132.1). Up to 5.8x single-stream on copy-heavy work, byte-identical output. The advisory ships in
plan. - Business validation: the recommended 2.5-bit 30B scored 40/40 machine-checked tasks against a pre-staked bar; a four-model capability ladder (52 executable predicates, T4 designed to fail today's models) ships in the repo.
- A2A, staked: depth-aware vs uniform Q2_K at equal bytes (+0.48%): -13.2% perplexity, -39.5% median KLD, +5.1 pts same-token, +6.6% tok/s - with the speed-invariance miss published as a miss.
- Two corrections of our own published numbers against primary sources (E-12 byte model; the Gemma-26B "C-02 violation" dissolved by its own GGUF header), prereg #92 killed by its own gates, probe workdir/exit-code fixes, 154 tests.
Every claim links to a pre-registration with its verdict - hits and misses at the same size.