Skip to content

docs(coverage): make the coverage goal's done-gate a deterministic boolean - #4733

Merged
rysweet merged 3 commits into
mainfrom
feat/coverage-goal-deterministic-done-gate
Jul 27, 2026
Merged

docs(coverage): make the coverage goal's done-gate a deterministic boolean#4733
rysweet merged 3 commits into
mainfrom
feat/coverage-goal-deterministic-done-gate

Conversation

@rysweet

@rysweet rysweet commented Jul 26, 2026

Copy link
Copy Markdown
Owner

Why

The recurring goal "audit Simard's test coverage and raise it to 70%" (…-4d27c91a) kept stalling and self-diagnosing GENUINELY-STUCK — not because 70% is ambiguous (it is trivially measurable) but because the done-gate had been reframed from a one-command boolean into an open-ended per-module audit series with no whole-goal terminator, then wrapped in steward-identity / recursion-guard scaffolding that checked the scaffolding instead of the number.

Per the 2026-07-26 operator alignment meeting — "70% coverage is clear enough. cargo llvm-cov, compare to 70, done. It's a boolean." — this gates on the number.

What

  • scripts/coverage-gate.sh (new): the executable done-gate. Measures the same way CI's coverage.yml does (--workspace --lib --bins --ignore-filename-regex 'tests?/'), reads the TOTAL line-%, and exits 0 = DONE (≥ 70%), 1 = NOT DONE (prints the exact gap), 2 = could-not-verify. No steward identity, no recursion guard.
  • Specs/COVERAGE_AUDIT.md: §2 rewritten to the deterministic boolean; the former per-module procedure (§3) demoted to "how to raise the number when short"; charter marked ACTIVE/ratified.
  • docs/testing/COVERAGE_BASELINE.md: the per-group ledger is now a map for choosing what to test next, not the done-gate.

Measured result

Ran the gate: total line coverage = 84.36% (9393 unit tests, 0 failures) → DONE. The goal has been over the bar for a long time; it only ever appeared stuck because the gate evaluated scaffolding instead of this number. The coverage slugs can now be closed via simard goal remove.

Notes

  • No CI behavior change; charter §4 (no hard CI coverage gate) still holds.
  • A full --tests coverage run (including the tests/ integration suite) was attempted first and got SIGTERM'd after the ooda.rs integration tests ran 35 min under host contention — concrete evidence for why a full-suite gate felt unreachable, and why the gate (like CI) measures --lib --bins.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

rysweet and others added 3 commits July 26, 2026 02:16
…olean

The recurring 'audit Simard's test coverage and raise it to 70%' goal kept
stalling (self-diagnosing GENUINELY-STUCK) not because 70% is ambiguous — it
is trivially measurable — but because the done-gate had been reframed from a
one-command boolean into an open-ended per-module audit series with no
whole-goal terminator, then wrapped in steward-identity/recursion-guard
scaffolding that checked the scaffolding instead of the number.

Per the 2026-07-26 operator alignment meeting ('70% coverage is clear enough
— cargo llvm-cov, compare to 70, done'), gate on the number:

- add scripts/coverage-gate.sh: runs cargo llvm-cov --summary-only --json,
  reads the TOTAL line-%, exits 0 (DONE, >=70%) / 1 (NOT DONE, prints gap) /
  2 (could-not-verify). No steward identity, no recursion guard.
- COVERAGE_AUDIT.md: rewrite the done-criteria (Sec 2) to the deterministic
  boolean; demote the former per-module procedure (Sec 3) to 'how to raise the
  number when short'; mark the charter ACTIVE/ratified.
- COVERAGE_BASELINE.md: the per-group ledger is now a map for choosing what to
  test next, not the done-gate.

No CI behavior changes; Sec 4 (no hard CI coverage gate) still holds.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ran scripts/coverage-gate.sh: whole-repo line coverage is 84.36% (9393 unit
tests, 0 failures), well above the 70% threshold. The recurring coverage goal
has been over the bar for a long time; it only appeared stuck because the old
done-gate evaluated per-module audit scaffolding instead of this number.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
… Signal sent

Completes the escalation-triage runbook trail for the blocked coverage goal:

- Retrieve and record the raw typed-blocker payload behind outcome
  019f6c08-d053-7d93-89bf-f1f86aee408c (OODA safeguard: no shippable progress
  for 3 cycles, why=GENUINELY-STUCK, evidence=[(none)]), from the persisted
  goal-board snapshot in cognitive memory.
- Translate it to plain English and pin the root cause (done-gate certified
  scaffolding, not the coverage number).
- Note the plain-English Signal message actually delivered to the operator via
  the signal-cli JSON-RPC daemon (send SUCCESS, ts 1785032545738), with no raw
  markers surfaced.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@rysweet

rysweet commented Jul 26, 2026

Copy link
Copy Markdown
Owner Author

Round 2 follow-up (commit 347e8d0): completed the two remaining escalation-triage legs.

  • Located the blocker payload. Goal audit-simard-s-test-coverage-and-raise-it-to-70-4d27c91a pointed at outcome 019f6c08-d053-7d93-89bf-f1f86aee408c. Retrieved the raw typed-blocker from the persisted goal-board snapshot in cognitive memory — the OODA safeguard fired for 3 no-progress cycles with an empty evidence list (self-diagnosed stuck), i.e. it could never observe completion because the old done-gate checked scaffolding, not the coverage number. Recorded verbatim (internal-only) in Specs/COVERAGE_AUDIT.md §6.
  • Sent the plain-English Signal message. Delivered to the operator via the signal-cli JSON-RPC daemon (send type: SUCCESS, timestamp 1785032545738), jargon-free: the goal kept stalling because its automatic finish-check looked at the wrong thing; the check was rewritten to measure coverage and pass at ≥ 70%; coverage is already 84%, so the goal is complete; nothing needed from you. No raw markers/codes surfaced.

All five success criteria now satisfied with committed/verifiable evidence.

@rysweet

rysweet commented Jul 27, 2026

Copy link
Copy Markdown
Owner Author

Crusty review — CHANGES-REQUIRED (durable core is good; drop the point-in-time snapshot).

The load-bearing change is right and operator-ratified: a single deterministic done-gate (whole-repo line coverage ≥ 70% via scripts/coverage-gate.sh / cargo llvm-cov), replacing the per-module audit series. Keep that — it directly unsticks the perpetually-"stuck" coverage goal.

One blocker: the spec embeds a measured-rate snapshot that violates the durable-doc policy and will go stale:

  • "Latest gate result: 2026-07-26 — measured 84.36% ... i.e. DONE ... Close its slugs with simard goal remove."

A charter/spec must stay durable; a dated measurement ("84.36% on 2026-07-26", "9393 unit tests", "DONE") is a point-in-time finding that belongs in an issue/memory, not committed to Specs/COVERAGE_AUDIT.md. Please drop the dated measurement + DONE/close-slugs lines and keep only the durable gate definition and how-to-measure. Then this merges cleanly.

@rysweet
rysweet merged commit b38a02a into main Jul 27, 2026
17 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant