fix(ci): validate promotion run identity - #723
Conversation
ytallo
added
the
no-ticket
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
skill-check — worker0 verified, 54 skipped (no docs/).
Four for four. Nicely done. |
📝 WalkthroughWalkthroughThe promotion workflow now identifies release runs by exact workflow paths. It accepts evidence from standard and repair-worker release workflows. Harness E2E validation requires a completed workflow-dispatch run with a successful conclusion. ChangesRelease validation
Estimated code review effort: 2 (Simple) | ~10 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches 💡 1🛠️ Fix failing CI checks 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/promote-worker.yml:
- Around line 201-203: Update the release-run and evidence-run validation checks
in the workflow to require both status == "completed" and conclusion ==
"success", in addition to the existing release.yml path checks. Ensure failed,
cancelled, in-progress, and queued runs are rejected before artifact validation,
while preserving any explicitly documented repair-exception behavior.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 7923e534-540e-46e1-8e2a-0ccbf2f96b50
📒 Files selected for processing (1)
.github/workflows/promote-worker.yml
| # GitHub exposes `run-name` through the API's `name` field, so the | ||
| # workflow file path is the stable identity for a Release run. | ||
| jq -e '.path == ".github/workflows/release.yml"' release-run.json >/dev/null || { |
There was a problem hiding this comment.
🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n -C 12 \
'status|conclusion|release_run_id|evidence_run_id' \
.github/workflows/promote-worker.yml \
.github/scripts/release_candidate.pyRepository: iii-hq/workers
Length of output: 23610
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== Release run and evidence run validation sections =="
sed -n '190,330p' .github/workflows/promote-worker.yml
echo
echo "== release_candidate.py relevant sections =="
sed -n '1,190p' .github/scripts/release_candidate.py
echo
echo "== release.yml workflow dispatch/branch info candidates =="
sed -n '1,140p' .github/workflows/release.yml || true
sed -n '140,260p' .github/workflows/release.yml || trueRepository: iii-hq/workers
Length of output: 24848
🏁 Script executed:
#!/bin/bash
set -euo pipefail
script="$(fd -a 'promote-worker.yml|release_candidate.py|release.yml' .github | grep 'promote-worker.yml\|release_candidate.py\|release.yml' | sort)"
python3 - <<'PY'
import json, pathlib, re
workflow = pathlib.Path(".github/workflows/promote-worker.yml").read_text()
checks = re.finditer(r"jq -e '(.*?)' [A-Za-z_-]+\.json >/dev/null", workflow, re.S)
print("release_run path-only check in promote-worker.yml:", ".github/workflows/p**romote-worker.yml" if any('RELEASE_RUN_ID' in str(workflow[max(0, m.start()-300):m.end()+300])) else False)
for m in checks:
text = m.group(1).replace("\\n", "\n").replace("\\'", "'")
if '.path == ".github/workflows/release.yml"' in text or 'Path to release candidate' in text:
print(f"FOUND at char {m.start()}; contains status/conclusion? status={'.status' in text}; conclusion={'.conclusion' in text}")
print(text.replace("\\n", "\n"))
# Extract candidate evidence JSON payloads from release_candidate.py to show that build has no run status inputs.
print("\n--- release_candidate.py build evidence fields ---")
script = pathlib.Path(".github/scripts/release_candidate.py").read_text()
m = re.search(r"def build_evidence\(args: argparse\.Namespace\) -> dict:\n(.*?)\n return \{\n \"schema_version\".*?\n \}", script, re.S)
print(m.group(0) if m else "not found")
print("contains run status/conclusion fields:", bool(re.search(r"(^|\s)(status|conclusion)[:\s]", m.group(0) or "")))
PYRepository: iii-hq/workers
Length of output: 260
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== jq gates around release-run.json and evidence-run.json =="
rg -n "release-run\.json|evidence-run\.json|Path to release candidate|jq -e" .github/workflows/promote-worker.yml
rg -n "status|conclusion" .github/workflows/promote-worker.yml .github/scripts/release_candidate.py
echo
echo "== release_candidate evidence build fields =="
python3 - <<'PY'
from pathlib import Path
import re
script = Path(".github/scripts/release_candidate.py").read_text()
m = re.search(r"def build_evidence\(args: argparse\.Namespace\) -> dict:.*?return \{(.*?)\n \}", script, re.S)
print(m.group(0) if m else "not found")
if m:
body = m.group(0)
print("contains run status/conclusion fields:", bool(re.search(r"(^|\s)(status|conclusion)=\s", body)))
PY
echo
echo "== release.yml result gating =="
rg -n "needs\\..*\\.result|failure\\(|cancelled\\(|promotable|dry_run|interface_smoke|setup.outputs|github.event" .github/workflows/release.ymlRepository: iii-hq/workers
Length of output: 10926
Reject failed or incomplete Release run evidence.
Lines 201-203 and 223-224 only match workflow paths. A failed, cancelled, in_progress, or queued RELEASE_RUN_ID or EVIDENCE_RUN_ID can still have an active/downloadable GitHub artifact; that artifact can be read by release_candidate.py validate, where any candidate_ready: true / promotable: true payload is accepted so long as its internal gate results are success.
Require status == "completed" and conclusion == "success" for both release-run checks, unless this is explicitly intended as a repair exception.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/promote-worker.yml around lines 201 - 203, Update the
release-run and evidence-run validation checks in the workflow to require both
status == "completed" and conclusion == "success", in addition to the existing
release.yml path checks. Ensure failed, cancelled, in-progress, and queued runs
are rejected before artifact validation, while preserving any explicitly
documented repair-exception behavior.
1 participant
Summary
namefield as a stable workflow identityRoot cause
GitHub returns the configured
run-namein the run APInamefield and returns workflow paths without an@refsuffix. Promotion required the static workflow name and suffixed paths, so a valid Release run failed before candidate evidence could be checked.Impact
Promotions can revalidate current Release and Harness E2E runs while still rejecting runs from any other workflow.
Validation
promote-worker.ymlas YAML3105995391231063151060git diff --checkSummary by CodeRabbit