Harden agent relay workflow authoring guidance#42
Conversation
|
Caution Review failedPull request was closed or merged during review 📝 WalkthroughWalkthroughThis PR standardizes mandatory sequential Claude→Codex fresh-eyes review/fix loops across workflow skills. Version numbers are bumped across README, prpm.json, and skill documentation to reflect substantive updates to the create-workflow command, relay-80-100-workflow, and writing-agent-relay-workflows guidance. The changes strengthen verification gates, evidence-based repair cycles, deterministic acceptance ordering, and commit/PR boundary gating. ChangesFresh-Eyes Review/Fix Loop Pattern Implementation
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@skills/writing-agent-relay-workflows/SKILL.md`:
- Around line 1358-1364: The acceptance step 'acceptance-after-codex-review'
currently just runs typecheck/tests and can pass even if an earlier artifact has
status BLOCKED_NO_COMMIT; update this step to perform a pre-check for blocker
artifacts (those marked BLOCKED_NO_COMMIT) before running the tests—e.g., add a
short command or script invocation at the start of the step that queries the
artifact store for any BLOCKED_NO_COMMIT entries (or checks the output of the
'codex-fix-final' dependency) and fail the step immediately with a clear message
if any are found, so the step 'acceptance-after-codex-review' cannot succeed
when blocker artifacts exist.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: c3b672d2-ce0d-4101-ba9f-22046a638562
📒 Files selected for processing (5)
README.mdcommands/create-workflow.mdprpm.jsonskills/relay-80-100-workflow/SKILL.mdskills/writing-agent-relay-workflows/SKILL.md
There was a problem hiding this comment.
Devin Review found 2 potential issues.
⚠️ 2 issues in files not directly in the diff
⚠️ Codex review step references stale pre-Claude-fix verification output via {{steps.verify-final.output}} (skills/writing-agent-relay-workflows/SKILL.md:388-398)
In the Conversation shape example, the codex-review step injects {{steps.verify-final.output}} into the Codex reviewer's task. However, verify-final runs before the entire Claude review/fix loop (claude-review → claude-fix → claude-review-final → claude-fix-final). If the Claude fixer modified source or test files, the verification output is stale and no longer represents the current codebase state. The same issue appears in the standalone "Mandatory Claude-Then-Codex Review/Fix Loops" pattern at skills/writing-agent-relay-workflows/SKILL.md:1325. In contrast, the Pipeline shape example correctly omits this stale reference. Since this skill document is the canonical template that agents copy when generating workflows, every generated conversation-shape workflow will propagate stale evidence to the Codex reviewer.
⚠️ Conversation and standalone acceptance steps missing BLOCKED_NO_COMMIT guard (skills/writing-agent-relay-workflows/SKILL.md:426-432)
The Pipeline shape acceptance step correctly checks test ! -f .workflow-artifacts/my-workflow/BLOCKED_NO_COMMIT.md before proceeding (skills/writing-agent-relay-workflows/SKILL.md:210). However, the Conversation shape verify-after-review step (line 426-432) and the standalone pattern acceptance-after-codex-review step (skills/writing-agent-relay-workflows/SKILL.md:1358-1364) only run npm run typecheck && npm test without checking for BLOCKED_NO_COMMIT.md. If the codex-fix-final step wrote a BLOCKED_NO_COMMIT.md because a finding couldn't be resolved (as instructed at lines 421-424 and 1350-1356), the acceptance step would still pass as long as tests pass — allowing commit/PR creation to proceed despite an unresolved blocker. This contradicts the skill's own Non-Negotiable Workflow Checklist item 7 (line 35): "write BLOCKED_NO_COMMIT with exact evidence and skip commit/PR creation."
View 4 additional findings in Devin Review.
Summary
Verification