Run the git stage against a real repository, and record what it did not cover - #184
Merged
Merged
Conversation
…ot cover Task 4.4 was marked done with a note saying it had not been executed. It has now been executed, against VeryComplexAndLongName/TestRepo, a scratch repository the owner provided. The repository was seeded with a workflow whose one job fails when a SHOULD_FAIL marker is present, so both outcomes could be produced on demand, and the run went through the real GitWrapper and the real PullRequestGateway - real git, real gh, no mocks. Passing path, PR #1: a branch with no upstream pushed through push("origin", branch); gh pr create opened the pull request and the number came from the URL it printed; waitForChecks returned pass; the merge succeeded and gh pr view reported MERGED. Refusing path, PR #2: the same sequence with the marker committed. waitForChecks returned {"state":"fail","reason":"check failed: Verify (failure)"}, the merge was never attempted, and gh pr view reported OPEN - the pushed branch and open pull request left for a person, as ADR 0014 requires. That pull request is deliberately still open as the evidence. Each of the three defects review found is one this run would have caught and no mocked test could. --json on gh pr create is refused by the real binary. SKIPPED appears only in real check output. A branch with no upstream is the only place a bare git push fails. The temporary test file that drove it was deleted rather than committed: it pushes to a live repository, and a suite CI runs is the wrong home for that. A new task 4.4a records what the run did not cover, unchecked. The chain's own wiring around the stage - shouldRunGitStage re-reading the per-change harness.json, the allowlist check and audit entry for each action, the completed event after the merge - needs archive to succeed first, which needs the disposable repository to be an OpenSpec project. Those junctions have unit coverage with a stubbed gateway, which is the right level for wiring and is not the same as having watched it. Better to say which is which than to let one checkmark imply both. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes
agentic-harness-git-stagetask 4.4, which had been marked done with a note saying it had not been executed.What ran
Against
VeryComplexAndLongName/TestRepo, the scratch repository the owner provided. Seeded with a workflow whose single job fails when aSHOULD_FAILmarker file is present, so both outcomes could be produced on demand.Driven through the real
GitWrapperand the realPullRequestGateway— realgit, realgh, no mocks.Passing path — PR #1,
MERGEDA branch with no upstream pushed through
push("origin", branch);gh pr createopened the pull request and the number came from the URL it printed; checks passed;gh pr viewreportedMERGED.Refusing path — PR #2, still
OPENThe merge was never attempted, and the pushed branch and open pull request were left for a person — as ADR 0014 requires. That pull request is deliberately still open in the scratch repository, as the evidence.
Why this run and not a mocked one
Each of the three defects review found is one this would have caught and no mocked test could:
--jsonongh pr creategh; only the real binary refuses itSKIPPEDread as a failureSKIPPEDappears only in real check outputgit pushafter an allowlistedgit push <remote> <branch>The temporary test file that drove it was deleted rather than committed: it pushes to a live repository, and a suite CI runs is the wrong home for that.
What it did not cover — new task 4.4a, unchecked
The chain's own wiring around the stage:
shouldRunGitStagere-reading the per-changeharness.json, the allowlist check and audit entry for each of the three actions, and thecompletedevent after the merge. Reaching those live needsarchiveto succeed first, which needs the disposable repository to be an OpenSpec project — a larger setup than the task described.They have unit coverage with a stubbed gateway (
under agent-sufficient plus allowlist, runs git push -> pr create -> merge,blocks a non-allowlisted git target before any push/pr/merge call, and the refusal cases). That is the right level for wiring, and it is not the same as having watched it. Better to say which is which than to let one checkmark imply both.Test plan
openspec change validate --strict agentic-harness-git-stage— validnpm run lint:english— passed🤖 Generated with Claude Code