fix: make release workflow resilient to re-runs [skip changelog] - #577
Conversation
- crates.io publish steps now tolerate "already exists" errors, allowing failed workflow re-runs to skip already-published crates - Version docs job creates a PR instead of pushing directly to main, respecting branch protection rules
There was a problem hiding this comment.
Pull request overview
Improves the release workflow’s ability to recover from failures by making crates.io publishing more tolerant to re-runs and updating the versioned-docs job to work with branch protection.
Changes:
- Make each
cargo publishstep tolerate “already exists” errors to support workflow re-runs. - Update the versioned docs job to push to a version branch and open a PR instead of pushing directly to
main.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| VERSION="${{ steps.version.outputs.version }}" | ||
| cargo publish -p agnix-rules 2>&1 || { | ||
| if cargo search agnix-rules --limit 1 | grep -qF "agnix-rules = \"$VERSION\""; then | ||
| echo "agnix-rules $VERSION already published, skipping" | ||
| else | ||
| echo "Publish failed and crate not found on crates.io" | ||
| exit 1 | ||
| fi |
There was a problem hiding this comment.
The “already published” detection relies on cargo search <crate> --limit 1 containing the exact version string. cargo search only returns the latest version, so this can false-negative when re-running older tags after a newer release exists (the older version is published but won’t appear in search output), and it can also false-negative during crates.io indexing lag. Consider instead keying off the cargo publish error output (match the known “already uploaded/exists” message), or querying the crates.io HTTP API for the specific version (/api/v1/crates/<crate>/<version>) with retries before deciding to fail.
There was a problem hiding this comment.
Fixed. Extracted into /tmp/publish-crate.sh helper that uses the crates.io HTTP API for reliable version detection instead of cargo search.
| - name: Publish agnix-rules | ||
| run: cargo publish -p agnix-rules | ||
| run: | | ||
| set -euo pipefail | ||
| VERSION="${{ steps.version.outputs.version }}" | ||
| cargo publish -p agnix-rules 2>&1 || { | ||
| if cargo search agnix-rules --limit 1 | grep -qF "agnix-rules = \"$VERSION\""; then | ||
| echo "agnix-rules $VERSION already published, skipping" | ||
| else | ||
| echo "Publish failed and crate not found on crates.io" | ||
| exit 1 | ||
| fi | ||
| } |
There was a problem hiding this comment.
This publish/skip logic is duplicated across all crates, which makes it easy for the steps to drift over time (and it’s already long in YAML). Consider generating a small helper script (e.g., /tmp/publish-crate.sh <name> <version>) and calling it for each crate to keep the behavior consistent in one place.
There was a problem hiding this comment.
Fixed. All 5 publish steps now call the shared /tmp/publish-crate.sh helper, keeping behavior in one place.
| BRANCH="docs/version-${{ steps.version.outputs.version }}" | ||
| git checkout -b "$BRANCH" | ||
| git commit -m "docs: version ${{ steps.version.outputs.version }} docs and update site data" | ||
| git push origin main | ||
| git push origin "$BRANCH" | ||
| gh pr create \ | ||
| --base main \ | ||
| --head "$BRANCH" \ | ||
| --title "docs: version ${{ steps.version.outputs.version }} docs" \ | ||
| --body "Auto-generated versioned docs snapshot for v${{ steps.version.outputs.version }}." \ | ||
| --label "documentation" |
There was a problem hiding this comment.
This docs step still isn’t resilient to re-runs: git checkout -b "$BRANCH" will fail if the branch already exists locally/remotely, and gh pr create will fail if a PR for that head branch already exists. To make re-runs safe, consider using git checkout -B (or git switch -C) and either updating the existing PR (detect via gh pr view --head "$BRANCH") or treating “PR already exists” as a non-fatal outcome.
There was a problem hiding this comment.
Fixed. Now uses git checkout -B (force-create), git push --force-with-lease, and checks for existing PRs via gh pr view --head before creating.
- Extract publish logic into /tmp/publish-crate.sh helper to avoid duplication across 5 crate publish steps - Use crates.io HTTP API (/api/v1/crates/<name>/<version>) instead of cargo search for reliable version detection on re-runs - Make docs branch/PR creation idempotent: use git checkout -B and detect existing PRs before creating new ones
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 1 out of 1 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if cargo publish -p "$crate" 2>&1; then | ||
| echo "$crate $version published successfully" | ||
| else | ||
| # Check crates.io API for the specific version (handles older tags, indexing lag) |
There was a problem hiding this comment.
The comment on line 265 mentions "handles older tags, indexing lag", but this check is specifically for detecting if a crate version is already published. The comment is misleading because the check happens during a publish failure, not as a general check for existence.
Consider clarifying: "Check if crate version already exists on crates.io (handles re-runs after partial failures)"
| # Check crates.io API for the specific version (handles older tags, indexing lag) | |
| # Check if crate version already exists on crates.io (handles re-runs after partial failures) |
There was a problem hiding this comment.
Fixed. Updated the comment to clarify it handles re-runs after partial failures.
| else | ||
| BRANCH="docs/version-${{ steps.version.outputs.version }}" | ||
| git checkout -B "$BRANCH" | ||
| git commit -m "docs: version ${{ steps.version.outputs.version }} docs and update site data" |
There was a problem hiding this comment.
Using git push --force-with-lease on line 488 is safer than regular force-push, but in this context where the branch is created fresh on line 486 with git checkout -B, the force-with-lease provides limited protection. If this is a re-run and the branch already exists remotely, force-with-lease will prevent overwriting if someone else pushed to it in the meantime.
This is correct behavior for the use case, but consider adding a comment explaining that force-push is intentional for re-runs.
| git commit -m "docs: version ${{ steps.version.outputs.version }} docs and update site data" | |
| git commit -m "docs: version ${{ steps.version.outputs.version }} docs and update site data" | |
| # Intentionally force-push to update the auto-generated docs branch on workflow re-runs. |
There was a problem hiding this comment.
Added a comment explaining force-push is intentional for re-runs of the auto-generated docs branch.
| --base main \ | ||
| --head "$BRANCH" \ | ||
| --title "docs: version ${{ steps.version.outputs.version }} docs" \ | ||
| --body "Auto-generated versioned docs snapshot for v${{ steps.version.outputs.version }}." \ |
There was a problem hiding this comment.
The PR body on line 497 could be more informative by mentioning that it includes rule docs regeneration and site data updates (as evidenced by the files added on line 481). This would help reviewers understand the full scope of changes.
Consider: "Auto-generated versioned docs snapshot for v${{ steps.version.outputs.version }}, including updated rule docs and site data."
| --body "Auto-generated versioned docs snapshot for v${{ steps.version.outputs.version }}." \ | |
| --body "Auto-generated versioned docs snapshot for v${{ steps.version.outputs.version }}, including updated rule docs and site data." \ |
There was a problem hiding this comment.
Updated to mention rule docs and site data updates.
| if gh pr view --head "$BRANCH" --json number --jq '.number' 2>/dev/null; then | ||
| echo "PR already exists for $BRANCH, updated with force push" | ||
| else | ||
| gh pr create \ | ||
| --base main \ | ||
| --head "$BRANCH" \ | ||
| --title "docs: version ${{ steps.version.outputs.version }} docs" \ | ||
| --body "Auto-generated versioned docs snapshot for v${{ steps.version.outputs.version }}." \ | ||
| --label "documentation" | ||
| fi |
There was a problem hiding this comment.
The PR existence check on line 490 suppresses all errors with 2>/dev/null, which means network failures or API errors will be treated the same as "PR doesn't exist" and will attempt to create a duplicate PR. While gh pr create will likely fail in those cases, it's more robust to check the exit code and stderr explicitly.
Consider: if gh pr view --head "$BRANCH" --json number --jq '.number' >/dev/null 2>&1; then
Or add error handling to distinguish between "not found" and other errors.
| if gh pr view --head "$BRANCH" --json number --jq '.number' 2>/dev/null; then | |
| echo "PR already exists for $BRANCH, updated with force push" | |
| else | |
| gh pr create \ | |
| --base main \ | |
| --head "$BRANCH" \ | |
| --title "docs: version ${{ steps.version.outputs.version }} docs" \ | |
| --body "Auto-generated versioned docs snapshot for v${{ steps.version.outputs.version }}." \ | |
| --label "documentation" | |
| fi | |
| PR_CHECK_ERR="$(mktemp)" | |
| if gh pr view --head "$BRANCH" --json number --jq '.number' >/dev/null 2>"$PR_CHECK_ERR"; then | |
| echo "PR already exists for $BRANCH, updated with force push" | |
| else | |
| if grep -qiE 'not[[:space:]]+found|could not find pull request|no pull requests' "$PR_CHECK_ERR"; then | |
| gh pr create \ | |
| --base main \ | |
| --head "$BRANCH" \ | |
| --title "docs: version ${{ steps.version.outputs.version }} docs" \ | |
| --body "Auto-generated versioned docs snapshot for v${{ steps.version.outputs.version }}." \ | |
| --label "documentation" | |
| else | |
| echo "Error checking for existing PR for $BRANCH:" | |
| cat "$PR_CHECK_ERR" | |
| rm -f "$PR_CHECK_ERR" | |
| exit 1 | |
| fi | |
| fi | |
| rm -f "$PR_CHECK_ERR" |
There was a problem hiding this comment.
Fixed. Now uses a temp file for stderr and distinguishes 'not found' from API errors before attempting PR creation.
- Clarify publish helper comment to describe re-run context - Add comment explaining force-push is intentional for re-runs - Improve PR existence check to distinguish "not found" from API errors - Improve PR body to mention rule docs and site data updates
Summary
Context
v0.14.0 release workflow had 4 failures:
agnix-rulespublished but indexing timed out, re-run failed becauseagnix-ruleswas "already exists"This PR fixes issues 1 and 2. Issues 3 and 4 will be tracked separately.
Test plan