v0.5.0 — QSR spec v0, screen harness, model-card emit

quantfit 0.5.0 — the CI-verifiable half of ROADMAP milestone 0.5: the measurement protocol becomes a versioned spec, and the tooling to run it at scale ships with it.

What is NOT in this release, stated plainly: the existence-proof hunt runs, the sensitivity-control run, the replication package, outreach, and the 0.5 GO/NO-GO decision. Those run against this tooling; none of them has started.

QSR spec v0 (spec/qsr-v0.md)

The durable asset the roadmap is built around: Quantization Safety Regression, v0, as a versioned protocol document. Paired-diff protocol; engine rules including the same-binary GGUF mandate; schema-v2 provenance field by field; statistics (at-risk denominators, Wilson intervals with method and sidedness disclosure, minimum detectable effect, the exit-code CI contract); per-axis screen aggregation; resource-stated hardware caps; the determinism canary; sensitivity-control conditionality labeling; and versioning rules. Every numeric claim in it was verified by executing the shipped code, and the document states honestly which sections are code-enforced at which symbol versus normative on publication practice. The CLI is the spec's reference implementation, not the other way around.

quantfit screen --targets targets.json --out DIR

Runs the paired diff sequentially over a manifest of quants and writes one schema-v2 drift report per target plus screen-summary.json.

• Bounds are per-stratum AND per-axis. Each axis has its own at-risk denominator, so a dangerous-axis flip on a target whose over-refusal axis was unmeasurable still enters the dangerous-axis bound instead of vanishing from the headline number. Strata are never pooled; there is no screen-wide rate and no field to put one in.
• Flagged, not found. n_regressed counts judge-flagged flips; n_regressed_human_verified is reported separately, and every flagged row carries human_verified: null until a human reads the pair. The judge is uncalibrated, so an unverified flag is a candidate.
• Conditionality is machine-carried. The manifest takes a sensitivity_control block (pass / fail / unmeasurable / not_run; absent means not_run). Any status but pass stamps ROADMAP 0.5's literal label — "conditional on undemonstrated detection sensitivity" — into every bound's conditionality field.
• A screen survives its targets. Per-target operational failures — gated repos, missing files, mispaired architectures, network and disk errors: the same (RuntimeError, OSError) class the CLI maps to exit 2 — become rows, not screen deaths. Target names are collision-checked case-insensitively so a Windows or macOS filesystem cannot silently merge two reports. Unknown manifest keys are refused at both levels.
• Exit codes mirror verify-safety: 0 clean, 3 a flagged regression, 4 an axis where nothing was measured, 2 operational.

quantfit emit model-card --report drift.json

Renders any schema-v2 report as a paste-ready model-card section: the verdict verbatim, both axes with Wilson CIs and MDE (zero-flip rates withheld exactly as verify-safety prints them), the full provenance chain including the same-binary hash statement, the scale-cap line, and the exact serve command — vllm serve for a transformers artifact, llama-server -m for a GGUF one. Missing binary hashes read as unverifiable rather than mismatched; an engine quantfit did not write gets an explicit note rather than a silently absent section; a tampered-but-schema-valid report exits 2 cleanly instead of tracebacking out of a string formatter.

Screen target list (screens/targets-0.5.json + curation audit trail)

15 targets — 12 GGUF pairs across 9 model families and 4 quantizer orgs, plus 3 transformers pairs. Every filename, revision, and size was verified against the HF API twice by independent agents and spot-checked again by hand. The curation file records the corrections rather than hiding them, including one candidate removed because its advertised "BF16 baseline" is an upcast of FP8-quantized weights — a defect the GGUF file-type guard structurally cannot catch, so curation now checks the base model's quantization_config. The maintainer's own anchor quant is disclosed as self-produced, with a rule to quote its stratum's bound both with and without it.

Also

• docs/sensitivity-control-v0.md: the Q2_K gross-degradation surrogate control, with its decision rule keyed on the report's unmeasurable_axes rather than the process exit code (a regression on the other axis shadows exit 4), and reproduction verified from provenance equality so a human/judge label divergence reads as the finding rather than a reproduction failure.
• Verdict strings now name every unmeasurable axis: an over-refusal-degenerate run no longer prints a plain clean verdict next to exit 4.
• New quantfit[awq] extra (transformers' AwqQuantizer needs gptqmodel); quantfit.run_screen / ScreenError / model_card_fragment exposed as lazy root exports.
• Ruff is now upper-bounded (>=0.16,<0.17) in CI and the dev extra — an unpinned lint tool broke a green branch mid-cycle, and the repo's standing rule is that churning dependencies get a cap that moves only after a validated run.

Tests

208 pass, all hermetic (65 new): the screen's exit contract, the manifest refusal matrix, per-axis denominators, conditionality stamping, and every model-card hardening case. CI covers Python 3.10–3.14 plus a clean-venv wheel install smoke on Ubuntu and Windows.

Full changelog: https://github.com/Sahil170595/quantfit/blob/main/CHANGELOG.md