Skip to content

The Phase Audit

Entuvo edited this page Jul 8, 2026 · 2 revisions

The Phase Audit

Run at every phase boundary (and before any release/merge milestone). The audit answers two questions: is the harness still intact, and has the work drifted from the plan, the scope, or reality. It exists because harnesses fail quietly — a rule softened during a refactor, a status hand-edited under deadline, a doc describing a world that no longer exists. Each check verifies disk reality, never a document's description of itself.

Verification doctrine

  • Run, don't read. "HARNESS.md says the self-test runs in CI" is a claim; running the self-test is the check.
  • PARTIAL is a first-class result. A check that couldn't run (missing environment, unavailable tool) reports PARTIAL with the reason — it never silently converts to a pass.
  • FAIL blocks the phase transition until fixed, or explicitly overridden by the user with a DECISIONS.md entry recording the override and its revisit_when.
  • An LLM-emitted number is never evidence. Counts, percentages, and "all tests pass" must come from executed command output captured in the report.

The seven checks

  1. Presence & wiring — every component in .claude/harness.json exists and is non-empty; every hook in .claude/settings.json resolves to a real file or carries the test ! -f missing-file guard; the check command runs. Orphans count both ways.
  2. Rules unweakened — recompute the SHA-256 of the delimited core-rules block in CLAUDE.md and compare to the manifest. Match → PASS. Mismatch with an authorizing DECISIONS.md entry → re-bless. Mismatch without one → FAIL, restore from template.
  3. Guards still fire — re-run the project-local self-test. It plants a fake violation for each guard and requires rejection. A guard that stopped rejecting fakes is hollow now, whatever it did last month.
  4. Status honesty — sample the statuses that advanced since the last audit; each must have evidence that exists, is non-empty, and came from an executed check. Regenerate any generated docs and diff against committed.
  5. Drift sweep — walk the catalog below.
  6. Retro & right-sizing — the same failure class twice is a pattern → propose a harness addition (backlog item, citing the incidents). A guard that only ever false-positived → propose retirement. Both are proposals, never silent changes.
  7. Report & handoff — write docs/audits/audit-YYYY-MM-DD.md, update ACTIVE.md, and tell the user the verdict with the one or two findings that matter most, in plain language.

The drift catalog

Drift type Drifted looks like
Scope creep Work with no PRD clause ref; diffs sprawling past the declared surface; cut-list items reappearing
Plan vs. code Steps marked done with no → verify: run; plan silently reordered; deviations unlogged
Copy-fork rot Requirement text pasted into stories/plans, now diverged from the PRD source
Assumption drift "X behaves like Y" stated from memory, no file:line citation
Stale unknowns Open toggles past their decide-by; work proceeding on un-flipped decisions
Decision expiry A decision whose revisit_when came true, still governing
Upstream divergence A patch outliving its retirement condition (a silent re-fork); upstream edits not in the ledger
Doc-consumer rot A doc updated by nobody, read by nobody, describing last month

Report verdict

Every audit ends in one of: PROCEED, PROCEED-WITH-FIXES, or BLOCKED (reason).

Cadence

Full seven checks at each phase boundary. An optional mid-phase spot check (after a rough week or a big merge) runs checks 3 + 4 + the Plan-vs-code row of check 5. Never batch audits — the value is catching drift while the diff is still small.


Canonical detail: references/drift-audit.md.

Clone this wiki locally