godaudits 2.18.0

Change safety as a separate evidence claim

godaudits 2.18.0 adds a deterministic, diff-scoped blast-radius review without changing repository audit scores or the AUDIT.json schema.

Added

• godaudits blast-radius plan, validate, apply, render, and evidence.
• Static impact routing across reverse references, public contracts, database schemas, dependency versions and patches, configuration flags, serialization, cross-language readers, lifecycle ordering, caches, generated artifacts, public entry points, and changed audit evidence.
• One or two explicit safety facts with a five-level proof ladder from assertion through running-app reproduction.
• Confirmed risks with separate likelihood and consequence, cleared risks with invalidation conditions, and a compiled blocked, unproven, or pass merge disposition.
• Proof-results binding to the exact base commit, head commit, and patch hash under review.
• Separate change-review and proof-results schemas, complete operating documentation, fixtures, and regression coverage.

Safety and correctness

• Static planning and proof import never execute recorded commands.
• Executable proof requires named authority, capability, environment, and isolation, and must match the authorization boundary.
• The before-merge proof must run the exact command chosen during planning.
• Shared redaction now covers bare GitHub, OpenAI-style, and Google API credential forms.
• Runtime verification no longer reads the unreachable finding.runtime_probe property from the closed AUDIT.json schema.

Verification

• 181 tests passed.
• 9 benchmark cases passed.
• 17 seeded detector cases passed.
• 5 evaluation suites passed.
• Pinned schema validators, official Agent Skills validation, package dry run, and runtime doctor passed.

See CHANGELOG.md and the change-safety contract for the complete details.

godaudits 2.17.0: evidence-backed copy signals

Reader-facing prose now follows the same evidence boundary as code findings. The runtime narrows review to high-confidence phrase candidates, while product, UX, and launch evaluators decide whether the complete sentence is specific, supported, and appropriate for its reader.

A phrase match never establishes AI authorship and never becomes a finding by itself.

Added

• Eight path-scoped copy signal kinds for puffery, promotional language, vague attribution, stock formulas, filler, stacked hedging, chatbot residue, and generic conclusions.
• Conservative reader-facing path rules with exclusions for tests, fixtures, release history, licenses, notices, generated prompts, and Markdown code blocks.
• A deterministic benchmark case, unit coverage, interpretation guide, migration guide, and third-party NOTICE.

Changed

• Product, UX, and launch checks now separate deterministic phrase discovery from contextual judgment.
• The audit report quality gate now requires concrete actors, mechanisms, locations, and observable results.
• EVIDENCE.json advances from schema 1.1 to 1.2 with the new signal kinds and an explicit interpretation limit.

Upgrade

Existing schema 1.1 evidence can still initialize an audit. Regenerate evidence before a re-audit or freshness check. See the migration guide in docs/MIGRATION-2.17.md.

Validation

• npm run check
• pinned release validators
• npm pack dry run
• runtime doctor
• GitHub CI

godaudits 2.16.0: gates that can fire

Two gates that could not fire, and the drift each one let through.

No behavior change. No check text, check count, dimension, or scoring behavior moves. An audit re-run under 2.16.0 scores exactly what it scored under 2.15.1. The catalog stays at 437 checks across 18 domains.

The detector gate could not catch the class it exists to catch

detector-gate.js validated its fixtures without a catalog, so the two rules that need catalog weights never ran over them, and four fixtures drifted into naming a routing check with no weighted owner. 2.15.1 fixed those fixtures by hand and recorded the gap; this closes the gap.

The obstacle was that a fixture is a single-domain fragment. Passing the full catalog produced 828 errors across 17 cases, none of them the rule that mattered, all of them completeness rules a fragment cannot satisfy by design: 476 ledger is missing, 289 missing applicability row, 34 version mismatches, 28 weight mismatches.

So validateAudit gains { catalog, fragment: true }. Fragment scope drops what a fragment cannot satisfy by construction (pinned versions, an applicability row per domain, a complete per-domain ledger, and the domain and per-check weights, since a weight normalizes across an audit the fragment does not contain) and keeps the conformance rules that still catch a real defect in one: an unknown check id, routing-check ownership, and a finding with no weighted owner in its domain. A full audit passes no flag and is held to everything.

Verified by injecting the regression: with one seed's ownerCheck removed the gate now turns red naming the exact rule, where before it passed silently.

The published weight table did not match the weights that score

references/intake.md published the legacy pre-profile weights from audit.js, which sum to 110, while catalog/profiles.json scored with weights summing to 100. Ten of eighteen domains disagreed: code-quality 10 against 9, build 10 against 9, database 8 against 7, architecture 8 against 7, product 7 against 6, ux 7 against 6, llm 6 against 5, deploy 6 against 5, observe 5 against 4, repo 5 against 4.

Nothing caught it because both files were internally consistent. It matters because the module is loaded by the model on every audit, and it states that the audit records the profile and weights so a reader can reproduce the overall score, which a reader following that table could not.

Added

• validateAudit(audit, { catalog, fragment: true }) for deliberately partial audits: a seeded corpus fixture, a single-domain excerpt, a hand-built test case.
• scripts/lint.sh profile-table compares the published balanced profile against catalog/profiles.json and asserts the published weights sum to 100.
• Three regression tests for fragment scope: that it drops the whole-audit rules, that it keeps routing ownership in both directions, and that it still rejects a check id the catalog no longer defines.

Fixed

• The detector gate validates catalog-aware and fragment-scoped.
• The balanced-profile table in references/intake.md matches the catalog.
• DOMAIN_WEIGHTS in lib/audit.js records that it is the catalog-free fallback, that it does not sum to 100, and that copying it into documentation is what produced the table above.

Full notes in CHANGELOG.md.

godaudits 2.15.1: seeded coverage for the system-design checks

2.14.0 and 2.15.0 shipped six checks with no regression coverage. No benchmark fixture held a cache, a queue, or a replica count; the detector corpus covered eight checks and none in architecture; and nothing under benchmarks/, evals/, or test/ referenced A-ARCH-24 through A-ARCH-29. A rename or a deletion would have stopped detecting them silently. This release closes that and the rule gap that allowed it.

No behavior change. No check text, weight, dimension, or scoring behavior moves. An audit re-run under 2.15.1 scores exactly what it scored under 2.15.0. Only pack_version changes; the catalog stays at 437 checks across 18 domains.

Added

• Six seeded fixtures, one per system-design check, each a small repository whose architecture record carries numeric targets and whose code contradicts exactly one of them: a price write that never invalidates the cached key (A-ARCH-24), a consumer that inherits its concurrency from the broker (A-ARCH-25), a published 5 minute RPO behind a once-per-day backup plan (A-ARCH-26), session and rate-limit state in module scope behind six replicas (A-ARCH-27), a rename confirmation read back from the async replica (A-ARCH-28), and a 99.95 percent target served by one replica (A-ARCH-29). The corpus goes from 11 cases to 17.
• ownerCheck on SEEDS. A routing check carries no weight of its own, so a seed whose failing check is one now names the weighted check in the same domain whose dimension the defect scores into, and the generated ledger carries both. That is what a real audit must do to satisfy the routing-ownership rule in lib/audit.js.
• A contributing rule. New ground rule 6: a new check ships with a seeded fixture, and a routing check's seed also names its owner. The rule exists because six checks reached two releases without one.
• Documentation for the authored corpus, which was previously undescribed: what it covers, one clause per fixture, no collateral, the routing owner convention, and the fact that the gate runs without a catalog so the two catalog-aware rules are checked by hand when a seed is added.

Fixed

Four pre-existing authored fixtures seeded a routing check and named no weighted owner, so each failed catalog-aware validation on both the routing-ownership rule and the weighted-owner rule. The gate never caught it because detector-gate.js calls validateAudit without a catalog and the corpus builder calls compileAudit the same way, so the fixtures backing the gate were exempt from a rule enforced on every real audit. A-SEC-30 now routes to A-SEC-3, A-CODE-25 to A-CODE-5, A-CODE-26 to A-CODE-3, and A-DB-24 to A-DB-15, each chosen from the control the seeded defect implicates rather than from a fixed table.

Release cadence

Cut inside the 30-day window in docs/RELEASE-POLICY.md, matching none of its three exceptions. benchmarks/ is listed in package.json files, so the corpus a consumer receives is part of the published artifact rather than repository-local scaffolding.

Full notes in CHANGELOG.md.

godaudits 2.15.0: closing the system-design mirror

Closes the mirror against godplans 1.12.0. That release added R-ARCH-21
through R-ARCH-24 to the plan-time architecture module: read consistency and
partition key per entity group, redundancy topology behind each availability
number, cache tiering with a staleness budget, and an overload posture per
entry surface. 2.14.0 had already covered caching, backpressure, recovery
objectives, and scale-out readiness from the audit side, so two obligations
were left with nothing checking them: what a read is allowed to see, and
whether an availability target has any redundancy standing behind it.

Two checks, 435 to 437. Both are audit-only and both are zero-weight routing
checks, so a repo's architecture score moves only through the dimension the
finding implicates, never through a new bucket. No scoring dimension changed
weight, so a 2.14.0 audit re-run under 2.15.0 scores the same unless one of the
two new checks fires.

Added

• A-ARCH-28 (audit-only, deep-trace, conditional on read replicas or a
  partitioned store) checks that read consistency was decided rather than
  inherited: replica-served paths tolerate the staleness they get, post-write
  reads on money, auth, and inventory resolve to the primary, and a table
  projected past single-node scale carries a partition key instead of an open
  growth curve. A balance or post-write confirmation served from a replica with
  no recorded staleness tolerance is High. Partition mechanics, index shape, and
  replication lag tuning stay F-DB's; this check cites them.
• A-ARCH-29 (audit-only, conditional on a deployed runtime the project
  operates) checks that availability claims have redundancy behind them: a
  component carrying an availability target runs more than one instance behind a
  health-checked router, or its single instance is recorded with an accepted
  downtime number, and the router removes an unhealthy instance instead of
  continuing to route to it. A stated target above the single-instance baseline
  running one replica with no recorded acceptance is High, as is a replicated
  service with no liveness or readiness probe. A-ARCH-11 remains the home for
  targets with no arithmetic at all; this one is about the means, not the math.
• Two remediation seeds: pin post-write reads to the primary and record the
  staleness budget, and put redundancy behind the availability target.
• Two anti-patterns hunted: replica-served truth (a balance or permission read
  pointed at a replica because it was faster, with no record of the staleness
  that buys) and availability theater (an uptime target with one instance behind
  it, or a replica set whose router has no probe to remove a broken member).
• Surface map gains redundancy and routing signals (replica counts, autoscaling
  ranges, liveness and readiness probes, load-balancer health checks, standby
  and failover declarations, multi-zone placement) and read-routing signals
  (reader endpoints and their call sites, ORM read/write splitting, partition
  and shard key declarations). Conditional sub-surfaces gain a deployed runtime
  the project operates and read replicas, so a library is never graded on
  replicas it never runs.

Changed

• The architecture mirror boundary line records R-ARCH-1..24 defined, up from
  R-ARCH-1..20, matching godplans 1.12.0. The boundary itself stays at 19: the
  audit-only range is numbered by arrival, not renumbered to track godplans,
  because a published check id has to keep resolving to the same check.
• The zero-weight routing note covers A-ARCH-24 through A-ARCH-29 and names
  where each lands: caching and backpressure into integration discipline,
  recovery objectives and read consistency into data architecture and
  invariants, scale-out readiness into component boundaries, redundancy behind
  an availability claim into NFR reality.

godaudits 2.14.0: system-design checks for load and failure

The architecture domain already audited the structural half of system design: shape, bounded contexts, invariant ownership, trust boundaries, decision records. It was thin on the half that only shows up under load or after a failure. Four audit-only checks close that gap.

Added

• A-ARCH-24 (caching contract), conditional on a caching layer existing: each layer records what it holds and for how long, every cached entity has a write path that invalidates or versions it, and no per-user or per-tenant value is served from a key carrying neither identifier. Unbounded in-process caches stay A-CODE-16's and the exploitable cross-tenant disclosure stays F-SEC's; this check owns the design decision and its missing invalidation.
• A-ARCH-25 (backpressure), conditional on async infrastructure: every consumer bounds its in-flight work, unbounded producer paths carry admission control, and overload has a recorded shed-or-degrade behavior rather than an out-of-memory kill. Retries with no circuit breaker into a saturated dependency are graded here; queue-depth alerting stays F-OBS's.
• A-ARCH-26 (recovery objectives), conditional on the project owning a durable store: a numeric RTO and RPO per store, a backup mechanism that satisfies the RPO arithmetically, and at least one restore actually performed and dated. An RPO shorter than the backup interval is arithmetic that already failed, the same treatment A-ARCH-11 gives latency budgets.
• A-ARCH-27 (scale-out readiness): no request-serving state in one process, no scheduled job firing once per replica by accident, no correctness depending on session affinity. The finding pairs the declared replica count with the state site that blocks it, so an architecture record promising horizontal scale over an in-memory session store is a finding rather than a claim.
• Four remediation seeds and four anti-patterns: accidental cache, unbounded intake, backup theater, scale-out fiction.
• Surface-map inventory for the new sub-surfaces, and three new conditional sub-surfaces so a project absent a cache, a durable store, or a replicated runtime records that reason instead of being graded on a surface it does not have.

Scoring

All four are routing checks. The catalog grows to 435 checks across 18 domains without shifting the scoring denominator. Findings score into the dimension of the surface they implicate: caching and backpressure into integration discipline, recovery objectives into data architecture and invariants, scale-out readiness into component boundaries.

Cost tier

A-ARCH-24, A-ARCH-25, and A-ARCH-27 are deep-trace: each needs a cross-module join (cache reads against every writer, consumer bounds against producer paths, in-process state against the deploy topology) that a targeted read cannot settle. A medium budget leaves them unknown, which lowers coverage rather than inventing a verdict. A-ARCH-26 is screening: its evidence is recorded objectives and dated drill artifacts.

Release cadence

Cut inside the 30-day window in docs/RELEASE-POLICY.md, matching none of its three exceptions. catalog/checks.json is a pinned contract: consumers resolve checks by pack_version, and the same version resolving to a different check count is worse than an off-cadence bump.

Full notes in CHANGELOG.md.

godaudits 2.13.0: wayfinding the remediation plan

Drawn from reviewing what the wayfinder skill (mattpocock/skills, MIT) could contribute. Its planning machinery has no place here: godaudits does not chart decision tickets, and the audit already produces the graph wayfinder assembles by hand. What was worth taking was its discipline about reading a graph back, which godaudits had never applied to its own remediation plan.

An audit ended with a list of phases and waves. That answers "what did the audit decide" and not "what can I start now", and it said nothing about what the audit consciously did not cover.

Added

godaudits wayfind AUDIT.json (and --format json) reports the destination, frontier, claims, blockers, fog, and scope boundary. It is a read: it compiles nothing and mutates nothing, so it stays correct on a half-written plan and the instant a task closes. The frontier is deliberately not written into computed, because a frontier committed into the audit record is stale the moment a task closes.

Three frontier rules the phase-and-wave listing could not express:

• A superseded dependency counts as closed. A replaced task will never complete, so treating it as an open blocker would strand its dependents forever.
• The final re-audit gate is the destination, not a member of the route. It depends on every task by validation rule, so counting it as an ordinary blocked task would put one permanently unreachable entry in every map and credit every task with the same meaningless unblock.
• Frontier tasks sharing a file are named as a concurrency conflict. Validation only rejects parallel file overlap inside a single wave; across waves it is legal state.

Four optional AUDIT.json fields, each validated:

Field	Rule
audit.destination	Prose naming what reaching the end looks like. An audit stating none is reported as stating none, not silently omitted.
task.claim	Owner and date. Rejected once the task is no longer open, where it would read as work in flight that is not.
check.question	On an unknown check, the question whose answer resolves it. Rejected on any resolved outcome.
not_yet_specified	Its domain must be applicable (fog only gathers toward the destination); any check it names must still be unknown (fog that graduated is cleared, not restated).

There is deliberately no rule requiring every unknown check to carry a question. A fresh audit initializes every check to unknown and a medium budget holds every deep-trace check unknown by design, so the gate would be unsatisfiable. The map reports the count instead.

docs/WAYFINDING.md records the borrowed disciplines, what was cut, and why.

Changed

The generated report leads the remediation plan with the destination and the frontier, both read before a task is chosen, and reports fog and scope in separate sections. Collapsing them would let a coverage gap read as a deliberate boundary, or a boundary read as a gap.

Task and finding references now carry their title with the id inside, replacing bare id lists on Depends on and Fixes. Check ids stay bare: their titles live in the catalog rather than in AUDIT.json, and the plan-aware mirror already uses that slot for the godplans R-id.

Fixed

A latent bug this version bump surfaced: the dogfood publication gate compiled retained historical artifacts against the current pack version, so every release after a dogfood publication would have failed it. The only escapes would have been editing the pinned engine_version (misattributing the run) or dropping the artifact. The version identity alone is now relaxed and every content rule stays live.

Compatibility

Every wayfinding field is optional and nothing was added to computed. Audits written before 2.13, including the committed dogfood and detector artifacts, validate unchanged and still produce a map.

The published auth0/node-jsonwebtoken dogfood report is deliberately not re-rendered: its AUDIT.json pins engine 2.12.0, so rendering it with this engine would misattribute the run. A retained report is evidence of a specific engine at a specific commit, not a live view.

Release cadence

This is a minor release outside the 30-day cadence in docs/RELEASE-POLICY.md and matches none of its three exceptions. It is cut because the change alters the shape of the generated report and the published AUDIT.json contract, and a documented contract change is clearer as a version than as unversioned drift on main.

Full changelog: v2.12.0...v2.13.0

godaudits 2.12.0

First measured-results release.

Evidence result:

• The first complete A-SEC-6 control versus installed-skill suite reached 15 of 15 pre-authored observations in both arms with no false positives. Measured skill lift is zero on this small suite.
• The installed-skill arm used 1.94 times the input tokens and 1.82 times the elapsed time.
• The first public OSS retrospective, auth0/node-jsonwebtoken before the CVE-2015-9235 fix, records zero hits, one miss, and zero adjudicated false positives. The broad missing allowlist was found, but the documented asymmetric-key to HMAC-key confusion path was missed.

Highlights:

• Retained paired outputs, model and harness attribution, attempt history, and append-only ground-truth corrections.
• Deterministic regrading separates causal hits, post-run truth, duplicate citations, false positives, and pre-inference technical failures.
• Public dogfood packages retain source evidence, compiled audit state, generated report, run metadata, redacted transcript, hashes, ground truth, unknowns, and escalation leads.
• Version and release documentation now publish null and negative evidence without converting near hits after disclosure.

Validation:

• 140 unit tests
• 8 benchmark cases
• 5 evaluation suites
• official Agent Skills reference validator
• JSON Schema 2020-12 validation
• npm package dry run
• ASCII, catalog, prompt, schema, and zero-dependency gates

See ACCURACY.md and CHANGELOG.md for the full record.

godaudits 2.11.0

Measurement, evidence, and audit economics

This release makes accuracy claims harder to manufacture and audit scope harder to overstate.

Highlights

• Adds a versioned accuracy program targeting the highest-weight check in all 18 domains, with pre-authored ground truth, clean controls, mandatory model and harness attribution, and matched with-skill versus without-skill arms.
• Adds godaudits import-tool for SARIF, Semgrep, ast-grep, Gitleaks, and OSV-Scanner. Imported output remains provenance-bound evidence leads, never automatic findings.
• Adds screening and deep-trace cost tiers. Medium is the focused default; full audits must be selected explicitly.
• Adds honest domain depth labels and escalation criteria. Security and build are deep-capable; remaining domains are screening-grade.
• Adds release cadence governance and a strict publication contract for recognizable OSS dogfood audits.

Accuracy status

The existing A-SEC-3 pilot remains five detected seeded defects, zero misses, and zero false positives on one clean control, with a Wilson lower bound of 0.5655. No matched control-versus-skill experiment has completed, none of the 18 expanded targets is fixture-ready, and no skill-lift value is claimed.

Validation

• 134 tests passed
• 8 benchmark cases passed
• 11 detector cases passed
• 5 evaluation suites passed
• Catalog, prompt, schema, ASCII, zero-dependency, and official validator gates passed

See CHANGELOG.md and ACCURACY.md for the complete scope and limitations.

godaudits 2.10.0

The detector gate now measures something real

2.9.0 built the measurement machinery and honestly reported nothing, because authored fixtures detect their own seeds by construction. This release supplies the missing input: recorded blind audit runs.

The experiment

Six seeded fixture repositories graded from blatant to genuinely subtle (an unscoped lookup of medical PII; a service whose list and update bind the tenant while the by-id read does not; a scoping helper one later handler bypasses; a handler that loads unscoped and filters afterward; an export ignoring the predicate its siblings apply) plus one control with no seeded defect at all.

Ground truth was authored before any audit ran and kept outside the repositories. Each auditor received only a repository path and the catalog's own A-SEC-3 definition: no ground truth, no defect count, no hint a control existed. Runs are captured verbatim, hits and misses alike.

5/5 seeded defects detected, 0 false positives on the control. A-SEC-3 clears the five-independent-audit floor and reports a measured detection rate of 1.0 with a Wilson lower bound of 0.5655 — five for five is not a perfect detector, and the artifact must not read as though it were. Checks carried only by authored fixtures still report authored-only and no rate.

The blind run corrected the ground truth

Ground truth for the post-filter case said High. The run said Critical and was right: A-SEC-3 keys severity on the data class ("Critical on PII, financial, or cross-tenant data"), and tickets across organizations are cross-tenant data. The original entry imported a compensating-control discount the rule does not contain, and had not weighed that the handler is fail-open when req.user is absent or that the 404/403 split is a cross-tenant existence oracle. Corrected in place, with the correction recorded rather than quietly rewritten.

Honest edges

Recorded findings carry Firm rather than the label the run reported, because the capture schema collected one citation each and one evidence path cannot support Certain: 2.8.0's corroboration invariant applied to real output. Recorded prose passes through the same secret redactor a real audit uses, which conservatively masked a shareToken: req.query.token code expression in remediation advice; the redactor was applied rather than loosened.

Scope

This measures detection on six fixtures, not a reliability estimate for unseen repositories, and no number here feeds a per-repo score. That boundary is enforced in code and stated in the artifact.

Tests 91 → 92; full gate green.