Skip to content

Fix Phase 3 behavioral-claim verification and command-substitution rejection - #93

Merged
dmccoystephenson merged 1 commit into
mainfrom
feature/localization-and-command-sub-fixes
Aug 1, 2026
Merged

Fix Phase 3 behavioral-claim verification and command-substitution rejection#93
dmccoystephenson merged 1 commit into
mainfrom
feature/localization-and-command-sub-fixes

Conversation

@dmccoystephenson

@dmccoystephenson dmccoystephenson commented Aug 1, 2026

Copy link
Copy Markdown
Member

Summary

Three small template-rule self-audit gaps, all localized to create-dev-loop.md prose and none requiring a CLAUDE.md edit:

Test plan

  • python3 scripts/check_docs.py — passes (every {{placeholder}} has a Step 4 row, README's "What it does" still 1:1 with Steps).
  • UNVERIFIED — manual /create-dev-loop fixture regeneration not run. This PR changes template surface (Phase 3 Localization verification, Phase 4 self-review/anchor-cannot-run text, Phase 6 commit form, Edge cases) that the CLAUDE.md "Testing changes" checklist would normally exercise by regenerating a fixture skill (e.g. against dpm-dev-loop or herald-dev-loop) and diffing. This session's sandbox restricts filesystem/tool access to only this checkout — ls ~/local-skills/, mkdir (even for a path inside this checkout), and cloning an external fixture all fail with "Claude Code may only [list/create] in the allowed working directories for this session." This is exactly the gap [self-audit] Phase 4's anchor-cannot-run guidance doesn't name session-sandbox path restriction as a trigger #90 (this PR) documents, and the same gap the prior cycle's PR Document gh pr create workaround for restricted fetch refspecs #89 hit. Per the anchor-cannot-run gating this PR itself edits: template surface changed → do not auto-merge, hand to a human who can run the fixture.
  • Confirmed the command-substitution rejection this PR fixes (Phase 4/6 prescribe a command-substitution heredoc form some harness classifiers reject outright #91) is live in this session too: a chained Bash command (... && echo "exit: $?") was rejected mid-cycle with the same class of classifier block the issue describes, before any template change was applied — direct evidence the prescribed heredoc form is unsafe to keep as-written.
  • Read the full diff (git diff) — every changed line is one of the three issues' named surface area; no unrelated formatting/renames.

Closes #90
Closes #91
Closes #92

No RESEARCH.md finding directly covers harness command-classifier behavior or session-sandbox path restriction (#90, #91) — these are execution-environment constraints observed empirically in live sessions, not covered by the cited literature. #92 extends the existing localization-verification instruction grounded in RESEARCH.md §3 (fault localization as the dominant uncontaminated-benchmark failure mode); no new citation needed since it sharpens an already-cited finding's application rather than introducing a new claim.


This PR description was drafted during a Gardener session (Stephenson-Software/gardener).

…docs

Phase 3's Localization verification only confirmed a named symbol
exists, not that the issue's description of its behavior was correct
past that point — add a third verification step so downstream doc
cycles don't paraphrase a wrong claim straight into project docs.

Phase 4 step 5 and Phase 6 prescribed a `--body "$(cat <<'EOF' ... EOF)"`
form that at least one harness's command classifier rejects outright
before it reaches the shell; switch both to `--body-file` / `-F` via a
Write-tool scratch file, and name that constraint next to the existing
scratch-file-handling rule. Also name the sandbox's session-scoped
filesystem restriction (gardener-dispatched sessions) as a trigger for
the Phase 4 anchor-cannot-run fallback, and add a retry note for
transient `gh` network errors to Edge cases.

Closes #90
Closes #91
Closes #92

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
@dmccoystephenson

Copy link
Copy Markdown
Member Author

Self-review rubric:

Summary: mechanically small, well-scoped fix for three self-audit gaps. The one real gap is that manual fixture-regeneration validation could not run in this sandboxed session — flagged UNVERIFIED and this PR is left open for a human (or a future cycle with fixture access) to run that check before merge, per this PR's own anchor-cannot-run rule.

@dmccoystephenson

Copy link
Copy Markdown
Member Author

Skip reasons for the remaining open issues not picked for this cycle (triage recorded 2026-08-01):

@dmccoystephenson
dmccoystephenson merged commit fc00a18 into main Aug 1, 2026
1 check passed
@dmccoystephenson
dmccoystephenson deleted the feature/localization-and-command-sub-fixes branch August 3, 2026 05:01
dmccoystephenson added a commit that referenced this pull request Aug 3, 2026
…docs (#93)

Phase 3's Localization verification only confirmed a named symbol
exists, not that the issue's description of its behavior was correct
past that point — add a third verification step so downstream doc
cycles don't paraphrase a wrong claim straight into project docs.

Phase 4 step 5 and Phase 6 prescribed a `--body "$(cat <<'EOF' ... EOF)"`
form that at least one harness's command classifier rejects outright
before it reaches the shell; switch both to `--body-file` / `-F` via a
Write-tool scratch file, and name that constraint next to the existing
scratch-file-handling rule. Also name the sandbox's session-scoped
filesystem restriction (gardener-dispatched sessions) as a trigger for
the Phase 4 anchor-cannot-run fallback, and add a retry note for
transient `gh` network errors to Edge cases.

Closes #90
Closes #91
Closes #92

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant