v3.4: template test of the two community deepseek-coder files; census template documentation corrected

quantcheck is a functional audit of quantized LLM files in public model registries: a 15-task screen run over every eligible tag of the official Ollama library plus 22 community files, 164-task confirmations of the flagged ones, recipe-matched rebuilds that trace five failures to uncalibrated quantization, controlled calibration experiments, a 66-tag miss audit, and a proposed acceptance gate.

What changed from v3.3

The two TheBloke deepseek-coder-6.7B community files (Q3_K_M and Q4_K_M) are template-dependent failures, not backend-dependent ones. On one RTX 4090 pod with the same Ollama release as the census, the files with no template reproduce the census answers byte for byte (5 and 6 of 15; 16 and 18 of 164), and with the deepseek-coder instruction template they score 13 and 15 of 15 (98 and 101 of 164), as they do on Metal. The files carry no chat template, so the runtime applied its default ChatML format, which differs from the model's documented instruction format. Records: rerun-2026-10/results-tpl/; serial scores: rerun-2026-10/serial/tpl_serial.jsonl.
PROVENANCE.md is corrected: the census community files ran with no TEMPLATE line (template_donor is null on all 22 rows); the seven referee files and the October reruns did get donor templates.
serial/numbers_regroup.json gains a template_test block. Every other value is unchanged: no census score, class, suspect, miss-audit value or attribution verdict changes.
Full list in RELEASE_NOTES_v3.4.md. The paper it supports is arXiv:2609.05881; the paper's next version carries this correction. No model files are included. Code: MIT. Data: CC BY 4.0.

v3.3: quantcheck audit dataset and tools, fully recomputable

quantcheck is a functional audit of quantized LLM files in public model registries: a 15-task screen run over every eligible tag of the official Ollama library plus 22 community files, 164-task confirmations of the flagged ones, recipe-matched rebuilds that trace five failures to uncalibrated quantization, controlled calibration experiments, a 66-tag miss audit, and a proposed acceptance gate. This release adds the per-task results of the 24 configurations the screen was tuned on, so every number in the analysis recomputes from the repository alone; see “Reproduce the paper’s numbers” in the README. The paper it supports is arXiv:2609.05881.

v3.0: audit dataset, rebuild and calibration evidence, and the quantcheck screen

Data, scripts and analysis behind the revised paper Broken on Arrival: Non-Functional Quantized LLM Artifacts in Public Model Registries and How to Catch Them (arXiv:2609.05881, version 2).

What changed from v1

• The v1 defect diagnosis is corrected. The five official Ollama tags that v1 reported as defective conversions are exact, uncalibrated llama.cpp quantizations, tensor-identical to rebuilds from the original checkpoints. They are still non-functional, and adding an importance matrix rescues most of them.
• Every reported score is a serial re-score (one EvalPlus evaluation at a time); see rerun-2026-10/serial/.

New in this release

• rerun-2026-10/: rebuilds, calibration, per-tensor comparisons, perplexity, generation-settings and miss-audit runs, the dated run plan, scripts and logs.
• rerun-2026-10/serial/numbers_serial.py reproduces the paper's numbers.

No model files are included. Code: MIT. Data: CC BY 4.0.