Skip to content

Correct stale CI scope, command surface, and repo owner in the docs - #107

Merged
dmccoystephenson merged 2 commits into
mainfrom
fix/ci-scope-and-trust-model-doc-drift
Aug 4, 2026
Merged

Correct stale CI scope, command surface, and repo owner in the docs#107
dmccoystephenson merged 2 commits into
mainfrom
fix/ci-scope-and-trust-model-doc-drift

Conversation

@dmccoystephenson

Copy link
Copy Markdown
Member

Summary

A Stage A documentation-accuracy sweep was run across every documentation source of
truth listed in CLAUDE.md. Four stale claims were found and corrected; two defects
that live in create-dev-loop.md's behavior or in agent-loaded configuration were
filed rather than changed here.

CI scope was understated in two places. .github/workflows/ci.yml runs two steps —
scripts/check_docs.py and python3 -m unittest discover -s tests. Both
CONTRIBUTING.md and .github/PULL_REQUEST_TEMPLATE.md described only the first,
so a contributor changing scripts/check_docs.py was never told that
tests/test_check_docs.py exists and needs new cases. CLAUDE.md already documents
the unit tests, and its documentation-sources table requires the PR template's
test-plan guidance to "match the current CI scope" and CONTRIBUTING.md's validation
checklist to match CLAUDE.md — both rows were out of date. The phrase "catches doc
drift, not behavior" was also sharpened to "not the template's behavior", since the
suite does test check_docs.py's own behavior.

SECURITY.md named only two of the four places a target-repo-derived command is
executed.
The trust model stated that COMPILE_CMD / TEST_CMD / LINT_CMD /
EXTERNAL_SIGNAL_CMD run "in its Phase 3 build-verification step and its Phase 4
external-signal anchor". {{TEST_CMD}} is also substituted into the Phase 8
post-rebase fence of the generated template, and {{REVALIDATE_INSTRUCTION}} — derived
from the same target-repo sources — points Phase 6 at that same check after every
review fix. Under-counting the execution surface understates the blast radius the
document exists to describe.

Both repositories referenced by README.md have moved to the Stephenson-Software
organization.
gh repo view resolves this repository as
Stephenson-Software/create-dev-loop and dmccoystephenson/gardener as
Stephenson-Software/gardener. The CI badge and the gardener link were updated to
the canonical owner. They resolved before only via GitHub's owner redirect, which would
silently point at the wrong project if a repository were ever created at the old path.
The two remaining dmccoystephenson links, in SECURITY.md and CODE_OF_CONDUCT.md,
point at the maintainer's personal profile and are correct as written; they were left
alone.

RESEARCH.md's "Last reviewed" date was bumped to 2026-08-04, matching the
precedent set by the sweep in PR #82. All eight findings, their confidence levels, and
their Implementations entries were re-checked and are accurate. Every one of the
17 arXiv citations plus the METR, Chroma, and Anthropic links was fetched and returned
200. The openai.com citation returns 403 — a WAF block against a non-browser
client, not a dead link, consistent with what PR #102 independently found. The PRs
merged since the last review (#89, #93, #95, #102, #103) each state in their own
descriptions that no finding applies, so no Implementations entry is missing.

Filed rather than fixed

Per the Stage A rule that a defect in behavior is filed and left for an implementation
cycle rather than changed silently under a docs-only PR:

Deferred backlog

Issue #87 was the only open issue at triage time and was not selected. A prior cycle
deliberately left it open, and its comment records why: the fix is an edit to a
generated skill file, which this loop is not authorized to make autonomously. The
underlying gap is tracked at its proper home as cdl-dev-loop#14.

Research grounding

No RESEARCH.md finding applies. This PR changes no phase definition, no placeholder,
no Step, and no template logic — it corrects factual claims in the supporting docs
about CI scope, command-execution surface, and repository ownership. Per CLAUDE.md's
research-grounding rule, that is stated explicitly rather than a citation being
stretched to fit.

Doc sync check

  • README.md's "What it does" Step list still matches create-dev-loop.md's Steps 1:1 — no Step was added, removed, or renumbered; verified mechanically by scripts/check_docs.py
  • Every {{placeholder}} added or changed has a corresponding Step 4 substitution-table row — no placeholder was touched; verified mechanically
  • RESEARCH.md updated — review date only, per above

Test plan

  • python3 scripts/check_docs.pyDoc consistency check passed.
  • python3 -m unittest discover -s testsRan 13 tests / OK
  • Every external link in every root *.md fetched; all 200 except the two known WAF 403s (claude.ai/code, openai.com) and one $OWNER placeholder inside a Step 7 code example
  • Each corrected claim re-read against its source: .github/workflows/ci.yml for the CI scope, the Phase 3 / 4 / 6 / 8 substitution sites in create-dev-loop.md for the command surface, gh repo view for both repository owners
  • Manual validation (/create-dev-loop against a real repo) — UNVERIFIED-not-applicable. The generated-skill anchor is required only when the PR changes the embedded template, the Step 4 substitution table, or Step 1/5–7 logic. This PR touches none of them: no file under create-dev-loop.md is modified at all. CI is the operative anchor here and covers every changed file.

This PR description was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener).

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@dmccoystephenson

Copy link
Copy Markdown
Member Author

Self-review rubric — scored adversarially against the diff and command output, not judgment.

Universal items

  • Scope: PASS — five files changed, each carrying a claim that was verified false against its source. README.md's owner change is the only item not implied by the phrase "CI scope"; it is named in the PR title and is the same category of defect (a documented fact that no longer holds), so it is declared rather than smuggled. No formatting sweep, rename, or comment churn beyond the one reflow noted below.
  • Tests-new: N/A — no script, function, or tool with testable behavior is added. scripts/check_docs.py and tests/test_check_docs.py are untouched; git diff --name-only origin/main...HEAD lists only Markdown.
  • Tests-fix: PASS (source-comparison form, not stash-and-run) — this repository's Phase 8 regression gate accepts a validation step for projects anchored on manual validation, and the empirical check for a doc-drift correction is the source read that reproduces the drift. Each correction was reproduced directly: .github/workflows/ci.yml contains two run: steps (python3 scripts/check_docs.py and python3 -m unittest discover -s tests -v) while the prior text named one; {{TEST_CMD}} appears at create-dev-loop.md:277 and :550 and {{REVALIDATE_INSTRUCTION}} at :424, while the prior text named two Phases; gh repo view returns Stephenson-Software for both repositories the prior links attributed to dmccoystephenson. A git stash experiment is not applicable — there is no production code path to revert.
  • Sibling structure: PASS — no new file is created. The edited passages retain each file's established voice and its ~72-column wrap.
  • Sibling renames: PASS — no identifier renamed. The two dmccoystephenson links left in SECURITY.md and CODE_OF_CONDUCT.md are deliberately not treated as siblings of the changed links: they address the maintainer's personal profile, which did not move, and changing them would have been the wrong rename.
  • Docs: PASS — every row of the CLAUDE.md documentation-sources table was walked. create-dev-loop.md unchanged and still internally consistent; README.md "What it does" still 1:1 with Steps 1–7; RESEARCH.md findings and Implementations entries re-verified; CONTRIBUTING.md and .github/PULL_REQUEST_TEMPLATE.md now match the real CI scope; SECURITY.md now matches the Step 2 read-list and the Step 4 execution sites; .github/ISSUE_TEMPLATE/*.md re-read and found to restate nothing that changed.
  • Issue resolution: PASS — no Closes #N is claimed, correctly. Step 2 names only the lowercase .github/pull_request_template.md, missing the uppercase spelling #105 and CLAUDE.md still links gardener under the old dmccoystephenson owner #106 are filed-not-fixed by design and remain open.
  • CI: PASSdoc-consistency green on head a707800, the final commit.

Repo-specific items

  • Placeholder table completeness: PASSgrep -oE '\{\{[A-Za-z_#/ ]+\}\}' create-dev-loop.md output is unchanged; create-dev-loop.md is not in the diff at all. Confirmed mechanically by scripts/check_docs.py, not from memory.
  • Fence escaping intact: PASS — vacuous but verified: no fence inside the embedded template is touched, since the file containing it is unmodified.
  • README/Step 1:1: PASS — no Step added, removed, or renumbered. check_docs.py's check_readme_steps_sync passes.
  • RESEARCH.md citation: PASS — the PR description states explicitly that no finding applies and gives the reason, rather than stretching one to fit.

Findings raised

One nit was found and fixed during this pass rather than left: the first edit to .github/PULL_REQUEST_TEMPLATE.md left a 48-column orphan line among ~72-column neighbours. It was reflowed in a707800. Flagged here because an all-PASS first draft is the expected shape of a self-review that has not looked hard enough, and this one did produce a real, if small, defect.

Two judgment calls are recorded for a human rather than silently resolved. First, SECURITY.md now says "Phase 8 post-rebase re-check"; the site is strictly the rebase fence inside the generated template's Edge cases section, which the Step 4 TEST_CMD row itself calls the "Phase 8 rebase fence" — the repo's own vocabulary was followed, but a reader scanning Phase 8 proper will not find the substitution there. Second, the README.md owner links were changed to the canonical org even though the old paths still resolve through GitHub's redirect; the argument for changing them is that a redirect can be shadowed by a repository later created at the old path, which would fail silently rather than visibly.

Anchor disclosure

Manual validation (/create-dev-loop against a real repository) was not run, and is recorded as UNVERIFIED-not-applicable. That anchor is required only when a PR changes the embedded template, the Step 4 substitution table, or Step 1/5–7 logic. None is touched — create-dev-loop.md is not modified. CI is the operative anchor for this change set and covers every file in it.

This review was posted as a plain PR comment rather than a formal Review object, and was drafted during a Gardener session (https://github.com/Stephenson-Software/gardener).

@dmccoystephenson
dmccoystephenson merged commit 7ff6184 into main Aug 4, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant