-
Notifications
You must be signed in to change notification settings - Fork 0
The Phase Audit
Run at every phase boundary (and before any release/merge milestone). The audit answers two questions: is the harness still intact, and has the work drifted from the plan, the scope, or reality. It exists because harnesses fail quietly — a rule softened during a refactor, a status hand-edited under deadline, a doc describing a world that no longer exists. Each check verifies disk reality, never a document's description of itself.
- Run, don't read. "HARNESS.md says the self-test runs in CI" is a claim; running the self-test is the check.
- PARTIAL is a first-class result. A check that couldn't run (missing environment, unavailable tool) reports PARTIAL with the reason — it never silently converts to a pass.
-
FAIL blocks the phase transition until fixed, or explicitly overridden by
the user with a
DECISIONS.mdentry recording the override and itsrevisit_when. - An LLM-emitted number is never evidence. Counts, percentages, and "all tests pass" must come from executed command output captured in the report.
-
Presence & wiring — every component in
.claude/harness.jsonexists and is non-empty; every hook the manifest marksenabledis present and wired; every hook in.claude/settings.jsonresolves to a real file or carries thetest ! -fmissing-file guard; the check command runs. Orphans count both ways. -
Rules unweakened — recompute the SHA-256 of the delimited core-rules block
in
CLAUDE.mdand compare to the manifest. Match → PASS. Mismatch with an authorizingDECISIONS.mdentry → re-bless. Mismatch without one → FAIL, restore from template. - Guards still fire — re-run the project-local self-test. It plants a fake violation for each guard and requires rejection. A guard that stopped rejecting fakes is hollow now, whatever it did last month.
- Status honesty — sample the statuses that advanced since the last audit; each must have evidence that exists, is non-empty, and came from an executed check. Regenerate any generated docs and diff against committed.
- Drift sweep — walk the catalog below.
- Retro & right-sizing — the same failure class twice is a pattern → propose a harness addition (backlog item, citing the incidents). A guard that only ever false-positived → propose retirement. Both are proposals, never silent changes.
-
Report & handoff — write
docs/audits/audit-YYYY-MM-DD.md, updateACTIVE.md, and tell the user the verdict with the one or two findings that matter most, in plain language.
| Drift type | Drifted looks like |
|---|---|
| Scope creep | Work with no PRD clause ref; diffs sprawling past the declared surface; cut-list items reappearing |
| Plan vs. code | Steps marked done with no → verify: run; plan silently reordered; deviations unlogged |
| Copy-fork rot | Requirement text pasted into stories/plans, now diverged from the PRD source |
| Assumption drift | "X behaves like Y" stated from memory, no file:line citation |
| Stale unknowns | Open toggles past their decide-by; work proceeding on un-flipped decisions |
| Decision expiry | A decision whose revisit_when came true, still governing |
| Upstream divergence | A patch outliving its retirement condition (a silent re-fork); upstream edits not in the ledger |
| Doc-consumer rot | A doc updated by nobody, read by nobody, describing last month |
Every audit ends in one of: PROCEED, PROCEED-WITH-FIXES, or BLOCKED (reason).
Full seven checks at each phase boundary. An optional mid-phase spot check (after a rough week or a big merge) runs checks 3 + 4 + the Plan-vs-code row of check 5. Never batch audits — the value is catching drift while the diff is still small.
Canonical detail: references/drift-audit.md.
Concepts