Description
Three improvements to the Claude AI rollback-safety workflow.
1. Split into a dedicated workflow file
The rollback-safety job used to live inside .github/workflows/ai_claude-orchestrator.yml, where it ran serially alongside the other Claude jobs. It has been extracted into its own standalone workflow file, .github/workflows/ai_claude-rollback-safety.yml, so it can run in parallel with the other Claude workflows (interactive, automatic review, backend reviewer). The new file follows the same pattern as .github/workflows/ai_claude-backend-reviewer.yml and gates on ./.github/actions/security/org-membership-check. The original job was removed from the orchestrator to prevent double execution.
2. Clear stale AI: labels on every new push
Added a new preflight-clear-stale-labels job that runs before the AI evaluation. On every PR push, any pre-existing AI: Safe To Rollback / AI: Not Safe To Rollback label is removed via gh pr edit --remove-label … || true. This fixes the issue where a PR could end up carrying both labels after a subsequent push flipped the verdict — each run now starts from a clean slate.
3. Skip AI evaluation when a human has already classified the PR
The same preflight job is gated off when the PR carries Human: Safe To Rollback or Human: Not Safe To Rollback, meaning the author has already acknowledged the rollback safety of the change. Implemented via contains(github.event.pull_request.labels.*.name, …) in the preflight if:; the skip cascades to claude-rollback-safety-check through needs: + default success() gating. Avoids redundant Claude API calls when a human has already weighed in.
Acceptance Criteria
Test Plan
Validate each improvement with a dummy PR against main. Each PR can be a trivial change (e.g., a README typo) — the behavior being verified is the workflow wiring, not the diff content.
Test 1 — Parallel execution (Improvement 1)
- Open a dummy PR with any trivial change.
- Open the Actions tab → the triggered run.
- Expected:
Claude AI Orchestrator and Claude AI Rollback Safety Check appear as separate workflow runs, starting at roughly the same time. The rollback-safety workflow no longer blocks or serializes with the orchestrator's other jobs.
Test 2 — Stale AI: label cleanup (Improvement 2)
- On an open dummy PR, manually add the label
AI: Not Safe To Rollback.
- Push a trivial commit that should clearly be classified as safe (e.g., a comment-only change).
- Expected:
preflight-clear-stale-labels runs and removes AI: Not Safe To Rollback before the AI job starts.
- After the workflow finishes, the PR carries only
AI: Safe To Rollback — never both labels.
- Repeat with the opposite stale label to cover the symmetric case.
Test 3 — Human override (Improvement 3)
- On an open dummy PR, add the label
Human: Safe To Rollback (and verify no AI: label is present).
- Push a trivial commit.
- Expected:
- Both
preflight-clear-stale-labels and claude-rollback-safety-check show status Skipped in the Actions tab.
- No
AI: label is added to the PR.
- No Claude API call is logged for this run.
- Repeat with
Human: Not Safe To Rollback to cover the symmetric case.
Cleanup
Close the dummy PRs without merging once the three scenarios pass.
Priority
Low
Additional Context
Files touched
.github/workflows/ai_claude-rollback-safety.yml — new
.github/workflows/ai_claude-orchestrator.yml — removed claude-rollback-safety-check job
Description
Three improvements to the Claude AI rollback-safety workflow.
1. Split into a dedicated workflow file
The rollback-safety job used to live inside
.github/workflows/ai_claude-orchestrator.yml, where it ran serially alongside the other Claude jobs. It has been extracted into its own standalone workflow file,.github/workflows/ai_claude-rollback-safety.yml, so it can run in parallel with the other Claude workflows (interactive, automatic review, backend reviewer). The new file follows the same pattern as.github/workflows/ai_claude-backend-reviewer.ymland gates on./.github/actions/security/org-membership-check. The original job was removed from the orchestrator to prevent double execution.2. Clear stale
AI:labels on every new pushAdded a new
preflight-clear-stale-labelsjob that runs before the AI evaluation. On every PR push, any pre-existingAI: Safe To Rollback/AI: Not Safe To Rollbacklabel is removed viagh pr edit --remove-label … || true. This fixes the issue where a PR could end up carrying both labels after a subsequent push flipped the verdict — each run now starts from a clean slate.3. Skip AI evaluation when a human has already classified the PR
The same preflight job is gated off when the PR carries
Human: Safe To RollbackorHuman: Not Safe To Rollback, meaning the author has already acknowledged the rollback safety of the change. Implemented viacontains(github.event.pull_request.labels.*.name, …)in the preflightif:; the skip cascades toclaude-rollback-safety-checkthroughneeds:+ defaultsuccess()gating. Avoids redundant Claude API calls when a human has already weighed in.Acceptance Criteria
.github/workflows/ai_claude-rollback-safety.ymlcreated with its ownsecurity-check,preflight-clear-stale-labels, andclaude-rollback-safety-checkjobsclaude-rollback-safety-checkjob removed from.github/workflows/ai_claude-orchestrator.ymlAI:labels on every new push so each run is evaluated from scratchHuman:label is present on the PRactionlintpasses on the new workflow fileTest Plan
Validate each improvement with a dummy PR against
main. Each PR can be a trivial change (e.g., a README typo) — the behavior being verified is the workflow wiring, not the diff content.Test 1 — Parallel execution (Improvement 1)
Claude AI OrchestratorandClaude AI Rollback Safety Checkappear as separate workflow runs, starting at roughly the same time. The rollback-safety workflow no longer blocks or serializes with the orchestrator's other jobs.Test 2 — Stale
AI:label cleanup (Improvement 2)AI: Not Safe To Rollback.preflight-clear-stale-labelsruns and removesAI: Not Safe To Rollbackbefore the AI job starts.AI: Safe To Rollback— never both labels.Test 3 — Human override (Improvement 3)
Human: Safe To Rollback(and verify noAI:label is present).preflight-clear-stale-labelsandclaude-rollback-safety-checkshow status Skipped in the Actions tab.AI:label is added to the PR.Human: Not Safe To Rollbackto cover the symmetric case.Cleanup
Close the dummy PRs without merging once the three scenarios pass.
Priority
Low
Additional Context
Files touched
.github/workflows/ai_claude-rollback-safety.yml— new.github/workflows/ai_claude-orchestrator.yml— removedclaude-rollback-safety-checkjob