v0.3.1 — DeepSeek-V4.1 data + reproducibility metadata

Adds the DeepSeek-V4.1-Flash-1017 measurements and makes "is this reproducible?" answerable per dataset.

Data

• deepseek-v41-flash-8x72-tp8.json — 8×72 GB, TP8+EP8, short 256/128 and long 8192/256 at C1–C32, three launches.
• deepseek-v41-flash-8x72-tp8-clean.json — the same matrix with --disable-radix-cache, extended to C64 and 32K prefill; every level reproduces within 0.2–3.4%.

Findings worth the read:

• C8 short-request is a reproducible dip (71–72 tok/s), not noise: the 128% spread in the cache-on run came from cache state.
• Prefix caching is worth ~34% of long-request throughput at C4 and ~48% at C8; short requests are unaffected.
• 32K prefill is the ceiling: 15.1 tok/s at C1, 22.7 at C4.

Failures

• F-17: DeepSeek-V4.1-Flash-1017 cannot run TP4 on 4×72 GB. The load OOMs in create_fp8_moe_weight_ at 70.74 GiB, with the log and the arithmetic (476 GiB ÷ 4 = 119 GiB per card).
• F-26: single-run short-request curves, now with the cache-on/cache-off contrast.

Reproducibility

• Every dataset declares a reproducibility block (full / partial / external-unverified), enforced by tools/check_repo.py.
• 3 full (both DeepSeek-V4.1 sets, MiniMax-M3.1), 3 partial (older data missing commands/clocks), 2 external-unverified.
• The MiniMax entry was upgraded by recovering the launch compose, the bench script with all 9,680 requests, and both image digests from the measurement host.

Also

• recipes/8x-72gb-pcie.md Case D; MODEL_MATRIX KV row now carries the measured 785 B/token next to the reported 2.6 KiB, with the 3.4× gap explained.
• PEP 621 metadata in pyproject.toml; CITATION.cff aligned to the release.

Checks: check_repo.py (parity, links, anchors, schema) · check_model_matrix.py · 42 unit tests · ruff — all green.

v0.3.0 — evidence audit

Every headline number in this release was re-checked against the data or the framework it came from. Where a conclusion outran its evidence, the conclusion changed.

Numbers corrected

• KV/token in MODEL_MATRIX — every fp8 row was 2x low (Qwen3.8-27B and Qwen3-32B 64 -> 128 KiB, Qwen3-235B-A22B 64 -> 94 KiB); Qwen3-8B was 128 -> 144 KiB. A derivation block now shows the geometry and tools/check_model_matrix.py re-derives the published rows in CI.
• planner.py PCIe ceiling — known PCIe SKUs fell into the "interconnect unknown" branch and capped TP at 2, so --gpus RTXPRO5000-72 --count 8 contradicted the repo's own TP8 measurement. It now proposes TP up to 8, or up to one switch with --group.
• TP8 vs TP4 — the 89% label was C8, not C1 (85.3%); TP8 used 2x the cards, and per GPU TP4 is 1.7x better.
• tok/s/W — two denominators produced 1.16 and 1.69 for the same run. The table now shows measured draw and limit-normalised side by side.

Conclusions corrected

• MoE quantization: the cited report's primary fix is W4A16 activations (5.9% -> 0 checksum failures). FP32 router weights are free but checkpoint-dependent (0/500 on QAD, 19.5% -> 4.0% pre-QAD). The chapter now says "activations first, router second".
• Two-shot all-reduce: the "threshold" claim came from a parameter name in a config whose value is off. Removed.
• TP=3 → TP=2 x PP=2: TP must divide heads and KV heads, and the documented card pair is already on PXB.
• NODE before SYS, FP8 needs Ada (SM89)+, gpu-memory-utilization is a total budget.

Added

• docs/09-failures.md — configurations that do not start, and ones that run but mislead.
• vLLM <-> SGLang flag mapping, with the three traps that make a copied config wrong.
• Latency-vs-concurrency table and SLO-based goodput guidance.
• Heading-anchor validation, benchmark-page parity, and SCHEMA fields for commands.*, clocks, image_digest.

Verification

check_repo.py (parity, links, anchors, schema) - check_model_matrix.py (5 models re-derived) - 42 unit tests - ruff clean - all green on CI.