Skip to content

v0.10.0: a defect found four times, fixed once

Choose a tag to compare

@rodlunt rodlunt released this 14 Aug 15:58
· 122 commits to main since this release
2b2d357

Eight changes since v0.9.1. The two that matter most are a defect this project kept re-finding, and a gap in what a run records about itself.

One fix instead of a fourth patch

The same defect had been found four times: a rendered summary that reads as a clean result when the underlying question was never answered. #100 (not-applicable verdicts rendering as "0 findings"), #122 (a could-not-run domain rendering as a bare zero), #184 (the evidence boundary reading "0 of 16" when nobody was asked), and #195 (the self-assessment limits block).

Each previous fix landed only where the defect was found, which is why the third shipped inside the block written to prevent the second. #204 routes every summary that states a count over a population through one helper that must return distinct text for three states, none found, none recorded, and N found, and adds a check that fails the suite when a new summary block is added without being classified. Verified against the real legacy run-state #184 was found on: Evidence boundary: 0 of 16 becomes 16 of 16 completed domains never recorded what they did not read.

A run now records where its time went

The server stamped its own clock exactly twice, at begin_run and at render_report. #206 adds domain_rules_fetched_at and domain_recorded_at, so a run knows when each domain's rules were first served and when its result was accepted. These are arrival stamps and not durations, and the field contract says so: in a run that fans out to a subagent per domain the domains overlap, so the gap between two stamps is not any domain's elapsed time.

The rest

  • #201: AUDIT.md now asks the auditor to record self_assessment. It never did, so the per-domain confidence column read "not reported" on every row while the README promised a finding from a shaky domain would not look identical to one from a solid one. A test now catches the next field added to the report but never requested by the protocol.
  • #203: a rule verdicted finding with no matching Finding is now rejected, closing the direction the consistency check missed. Consent flags are derived from TelemetryConsent.model_fields rather than a hand-maintained list, and docs/feedback.md names all nine consented sections instead of four.
  • #200: the test suite no longer makes about 180 live git ls-remote calls to github.com, dropping it from 3 minutes 24 seconds to 55 seconds, and letting it run offline.
  • #202: the recorded cost figures carry the tool version they were measured at, the WCAG 2.2 AA claim states which criteria are actually machine-checked, and the Closes #N policy carves out release and housekeeping pull requests.
  • #208: the reader-conclusions consent row no longer reads like a prompt for input the configuration page does not have.
  • #209: the README screenshots are captured against this build.

Upgrading

Change the tag in your registration command and re-register. There is no schema_version bump, so run-state files written by earlier builds still load and re-render.

Two behaviours changed for a caller. record_domain_result now rejects a rule verdicted finding that carries no Finding, and AUDIT.md asks for a field it did not previously mention. Both are tightenings, and both fail loudly with an actionable message rather than silently.