Skip to content

Docs: bring the audit tutorial and CATALOG up to what audit 0.2.0 actually does - #35

Merged
mmcky merged 2 commits into
mainfrom
docs-post-0.2.0-tidy
Aug 7, 2026
Merged

Docs: bring the audit tutorial and CATALOG up to what audit 0.2.0 actually does#35
mmcky merged 2 commits into
mainfrom
docs-post-0.2.0-tidy

Conversation

@mmcky

@mmcky mmcky commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Follow-through on #34. The tutorial is the procedure for run 2, and run 2 is the run that tests resume — so it is the one document that could not be left describing the pre-#34 behaviour.

Step 4 taught the resume rule that #34 replaced. It said to read findings.md and resume at the lowest number with no entry, which is exactly the single-block assumption defect 2 was about. It now describes partitioning issues.json by state and resuming each side independently, and it names the new failure mode worth watching for: resuming the open set correctly while re-doing the closed set from scratch. It also suggests interrupting during the closed pass if the chance comes, since that is the half that has never been checkpointed at all.

The same step quoted doctrine's old three-form citation listfile:line, a merged PR, or a tag — with no reachability rule, which is the drift Copilot caught inside doctrine on #34 surviving one file over. It now names the fourth form and, more importantly, the requirement that whatever is cited resolves on the ref the audit named.

Two review checks are strengthened where run 1 showed them weak. Check 3 passed a citation pointing at an unmerged branch, so it now says to resolve each sampled citation on the ref the header names, with git merge-base --is-ancestor for commits. Check 5 held in the report and broke in the checkpoint, so it now looks at findings.md as well as the output — closed entries reaching the catalog with no ## Closed block is precisely how run 1 passed that check in its report while failing it in its log.

CATALOG records the two severity-1 fixes. Its central caveat is unchanged and still true: resumability is untested, because run 1 was never interrupted.

reviews/audit-run-action-translation-2026-07-28.md is deliberately untouched — it is a dated record of what the doctrine said at the time, and correcting it would destroy the evidence that the defects were real.

Repo-level files only, so no plugin directory is touched and nothing bumps; check-version-bump.py agrees.

🤖 Generated with Claude Code

…ually does

The tutorial is the procedure for run 2, and run 2 is the run that tests
resume — so it was the one document that could not be left describing the
pre-#34 behaviour.

Step 4 taught the single-block resume rule that #34 replaced: read findings.md,
resume at the lowest number with no entry. It now describes partitioning
issues.json by state and resuming each side independently, and names the new
failure mode — resuming the open set correctly while re-doing the closed set
from scratch. The same step quoted the three-form citation list that doctrine §2
has outgrown, without the reachability rule that is the whole point of defect 1.

Two review checks are strengthened where run 1 showed them to be weak. Check 3
passed a citation that pointed at an unmerged branch, so it now says to resolve
each sampled citation on the ref the header names. Check 5 held in the report
and broke in the checkpoint, so it now looks at findings.md as well as the
output.

CATALOG records the two severity-1 fixes. Its central caveat is unchanged and
still true: resumability is untested, because run 1 was never interrupted.

Repo-level files only — no plugin directory touched, so no version bump.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates repository-level documentation to reflect the behavior and expectations of audit as of 0.2.0 (post-#34), especially around phase-2 resumability/checkpointing and the “citations must resolve on the named ref” rule.

Changes:

  • Update the audit tutorial’s Step 4 to describe per-state resuming (## Open / ## Closed) and the ref-resolvable citation requirement.
  • Strengthen two post-run review checks in the tutorial (resolve sampled citations on the named ref; verify closed-side checkpointing via findings.md).
  • Update CATALOG.md to note the two severity-1 fixes shipped in audit 0.2.0.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File Description
docs/tutorial-run-an-audit.md Updates Step 4 and review checks to match audit 0.2.0’s resume/checkpoint and citation-resolution behavior.
CATALOG.md Notes that the two severity-1 defects addressed in #34 are fixed as of audit 0.2.0.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread docs/tutorial-run-an-audit.md Outdated
Comment thread docs/tutorial-run-an-audit.md Outdated
Comment thread docs/tutorial-run-an-audit.md Outdated
…f the citation list

Copilot's three comments on #35, all on text this PR introduced.

Step 4 said the closed pass "has never been checkpointed at all" two sentences
before crediting #34 with checkpointing it. True of run 1, false of main. The
point is that it has never been *exercised*, which is what it now says.

The doctrine §2 link was hard-coded to blob/main while line 7 of the same file
links doctrine relatively, and developing-skills says to keep relative links
relative because CI checks they resolve. The AGENTS.md absolute-URL rule governs
links out of a plugin directory, since an installed plugin ships alone and
cannot resolve `../`; docs/ never ships alone, so relative is right here.

Review check 3 enumerated citation forms and omitted tags. Rather than sync what
had become the fourth copy of that list, the row no longer enumerates at all —
the same fix applied to doctrine §1 on #34, and what line 7 means by "points,
never restates". The list at line 79 is kept deliberately: it is the teaching
sentence and it links to §2 in the same breath.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@mmcky
mmcky merged commit 3dc9aeb into main Aug 7, 2026
5 checks passed
@mmcky
mmcky deleted the docs-post-0.2.0-tidy branch August 7, 2026 05:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants