Skip to content

godaudits 2.16.0: gates that can fire

Choose a tag to compare

@hannsxpeter hannsxpeter released this 04 Aug 04:57
· 6 commits to main since this release
d20bbbf

Two gates that could not fire, and the drift each one let through.

No behavior change. No check text, check count, dimension, or scoring behavior moves. An audit re-run under 2.16.0 scores exactly what it scored under 2.15.1. The catalog stays at 437 checks across 18 domains.

The detector gate could not catch the class it exists to catch

detector-gate.js validated its fixtures without a catalog, so the two rules that need catalog weights never ran over them, and four fixtures drifted into naming a routing check with no weighted owner. 2.15.1 fixed those fixtures by hand and recorded the gap; this closes the gap.

The obstacle was that a fixture is a single-domain fragment. Passing the full catalog produced 828 errors across 17 cases, none of them the rule that mattered, all of them completeness rules a fragment cannot satisfy by design: 476 ledger is missing, 289 missing applicability row, 34 version mismatches, 28 weight mismatches.

So validateAudit gains { catalog, fragment: true }. Fragment scope drops what a fragment cannot satisfy by construction (pinned versions, an applicability row per domain, a complete per-domain ledger, and the domain and per-check weights, since a weight normalizes across an audit the fragment does not contain) and keeps the conformance rules that still catch a real defect in one: an unknown check id, routing-check ownership, and a finding with no weighted owner in its domain. A full audit passes no flag and is held to everything.

Verified by injecting the regression: with one seed's ownerCheck removed the gate now turns red naming the exact rule, where before it passed silently.

The published weight table did not match the weights that score

references/intake.md published the legacy pre-profile weights from audit.js, which sum to 110, while catalog/profiles.json scored with weights summing to 100. Ten of eighteen domains disagreed: code-quality 10 against 9, build 10 against 9, database 8 against 7, architecture 8 against 7, product 7 against 6, ux 7 against 6, llm 6 against 5, deploy 6 against 5, observe 5 against 4, repo 5 against 4.

Nothing caught it because both files were internally consistent. It matters because the module is loaded by the model on every audit, and it states that the audit records the profile and weights so a reader can reproduce the overall score, which a reader following that table could not.

Added

  • validateAudit(audit, { catalog, fragment: true }) for deliberately partial audits: a seeded corpus fixture, a single-domain excerpt, a hand-built test case.
  • scripts/lint.sh profile-table compares the published balanced profile against catalog/profiles.json and asserts the published weights sum to 100.
  • Three regression tests for fragment scope: that it drops the whole-audit rules, that it keeps routing ownership in both directions, and that it still rejects a check id the catalog no longer defines.

Fixed

  • The detector gate validates catalog-aware and fragment-scoped.
  • The balanced-profile table in references/intake.md matches the catalog.
  • DOMAIN_WEIGHTS in lib/audit.js records that it is the catalog-free fallback, that it does not sum to 100, and that copying it into documentation is what produced the table above.

Full notes in CHANGELOG.md.