[review-retro] Review Retrospective — PR #664: review-on-open never completed before merge

[github-actions]

Review Retrospective — PR #664

PR: fix: pass drift data via step outputs — agent can't access activation filesystem
Merged: 2026-04-22T03:57Z by @PureWeen
Review runs: 0 automated reviews completed

---

What Happened

The review-on-open.agent triggered on PR open (03:34 UTC) but the PR was merged at 03:57 UTC — 23 minutes later. The review workflow requires 3 parallel sub-agents with adversarial consensus and has a 90-minute timeout. It could not complete in that window. Zero reviews or comments were posted before merge.

This was the third consecutive workflow PR (#653 → #662 → #664) iterating on the same instruction-drift workflow. Each went in without a completed automated review.

Skill Coverage

Skill	Relevant?	Referenced?	Gap?
gh-aw-guide	✅ Yes — workflow files changed	❌ Review did not run	⚠️ Structural timing gap
instruction-drift	✅ Yes — the skill being fixed	❌ Review did not run	⚠️ Structural timing gap
processing-state-safety	❌ No C# code changed	—	—
All other skills	❌ Not applicable	—	—

Review Accuracy

Metric	Count
True Positives	0
True Positives (Fixed)	0
False Positives	0
False Negatives	0 (review never ran)
Unresolved	1 (pre-existing, see below)

PR Changes Were Correct

The diff was sound:

• Lock file properly regenerated (frontmatter_hash updated, verified via GH_AW_STEPS_* env vars added to all agent invocation steps)
• issues: write removed from frontmatter is safe — compiler adds it to safe_outputs and conclusion jobs automatically; agent job only needs contents: read + pull-requests: read
• $GITHUB_OUTPUT template substitution pattern is correct
• Python setup step correctly added for python3 JSON parsing in activation job

Pre-Existing Gap (Not Introduced by This PR)

create-pull-request in instruction-drift.agent.md lacks github-token-for-extra-empty-commit: — per the gh-aw anti-patterns guide, GITHUB_TOKEN pushes do not trigger CI on agent-created PRs. Drift PRs currently land with no CI run. This pre-dates PR #664 and was not caught by any prior review.

Improvement Suggestions

1. Add status-comment: true to review-on-open.agent.md — posts a start/complete comment when the workflow triggers, signaling to the PR author that automated review is in progress. Without it there's no indicator that the 90-minute review was even triggered, so authors routinely merge before it finishes.

2. Branch protection: Consider requiring the Expert Code Review (auto) status check to pass before merging to main. This is the only robust solution to the timing gap for fast-merged infra/workflow PRs.

3. Fix github-token-for-extra-empty-commit on instruction-drift.agent.md's create-pull-request safe output — a PAT or App token is required to trigger CI on generated drift PRs. See the gh-aw anti-patterns table.

Generated by Review Retrospective for issue #664 · ◷

• expires on May 22, 2026, 4:09 AM UTC

[github-actions]

This issue is being closed as outdated. A newer issue has been created: #672

View newer issue

---

This action was performed automatically by the Review Retrospective workflow.