Skip to content

v1.2.0 — vacuous-test warn + verification-tier taxonomy

Choose a tag to compare

@akahkhanna akahkhanna released this 14 Jul 15:41
· 12 commits to main since this release

New Class-1 check vacuousTestFindings (warn-only, claim-gated, JS/TS only): an
added test whose body makes no call and does not throw provably cannot fail.
Scoped to be false-positive-free rather than broad — a delegated assertion, a
table-driven loop, and a "doesn't throw" smoke test all have a call and stay
silent; a genuinely empty/console-only body fires. Fires only on a fully-added,
brace-balanced block (edited-test bodies outside the hunk abstain).

Adversarial review (Fable) found four FP families in the first cut — a gap-blind
line join, unhandled regex literals, and the thesis hole that chai .should
getter chains and bare await are call-free failures — all fixed at root and
pinned; .should/await now count as actions, and a gap sentinel enforces the
fully-added-block invariant.

ROADMAP.md: the "deterministic vs needs-a-model" line is split into four
verification tiers (T1 artifact-held / T1.5 execution-evidence-in-transcript /
T2 empirical / T3 model), prompted by a reader who caught a real misfiling —
9d passes-only-the-visible-test and 8 regression-blind are T2 (a test run), not
the model layer. Also logs two known self-match FP families (Class-1 substring
scope, Class-9 own-source) as tracked deferred precision.

Self-check 596 -> 613; red-team 14/14; corpus unchanged (17/23 at-capture).

Co-Authored-By: Claude Opus 4.8 (1M context) noreply@anthropic.com

Full symptom → root-cause → fix → regression-test in FIXES.md (v1.2.0).