Skip to content

fix: handle race condition when PR is merged before agent job checks out branch#25581

Merged
pelikhan merged 2 commits intomainfrom
copilot/investigate-failure-issue
Apr 10, 2026
Merged

fix: handle race condition when PR is merged before agent job checks out branch#25581
pelikhan merged 2 commits intomainfrom
copilot/investigate-failure-issue

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 10, 2026

When a PR is merged within seconds of triggering a pull_request workflow, the webhook payload captures state: "open" — but by the time the agent job reaches "Checkout PR branch", the branch is already deleted. The existing isClosed guard reads only the stale payload state, so it falls through to the error path instead of the expected graceful closed-PR handling.

Changes

  • checkout_pr_branch.cjs: In the catch block, when the payload says the PR is "open" but checkout fails, re-query the GitHub API for the current PR state. If the PR is now closed/merged, emit a warning and return cleanly (checkout_pr_success: true) instead of calling setFailed.
// After checkout throws, before treating as a hard error:
const { data: currentPR } = await github.rest.pulls.get({ ... });
isNowClosed = currentPR.state === "closed";
// → treat identically to isClosed from payload

The API failure warning now includes the HTTP status code for easier triage.

  • checkout_pr_branch.test.cjs: Added 7 new tests in a "race condition - PR merged after workflow trigger" describe block covering: graceful handling when PR is merged mid-flight, still-failing when PR is genuinely open, API re-check failure with HTTP status in warning, HTTP status code included/omitted correctly, correct API call arguments, and the race condition on the gh pr checkout path (e.g. issue_comment events).

…R race condition

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/91e7d084-780b-4652-b435-d4833c7555d0

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan
Copy link
Copy Markdown
Collaborator

@copilot Update tests

…after trigger

Agent-Logs-Url: https://github.com/github/gh-aw/sessions/2f44d3d1-8b2e-4d87-bac6-6464a26b05ca

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 10, 2026

@copilot Update tests

Added 7 new tests in a "race condition - PR merged after workflow trigger" describe block (commit 243c151), covering:

  • PR merged after trigger → checkout failure treated as warning (not error)
  • PR still open when re-checked → still fails as error
  • API re-check itself fails (with HTTP status in warning)
  • HTTP status code included/omitted correctly
  • Correct PR number/repo passed to the API call
  • Race condition on the gh pr checkout path (issue_comment event)

All 48 tests pass (41 existing + 7 new).

Copilot AI requested a review from pelikhan April 10, 2026 05:23
@pelikhan pelikhan marked this pull request as ready for review April 10, 2026 05:24
Copilot AI review requested due to automatic review settings April 10, 2026 05:24
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes a race condition where a pull_request workflow can start with a stale "open" PR state in the webhook payload, but the PR is merged/closed (and branch deleted) before the agent job reaches checkout—causing an unnecessary hard failure.

Changes:

  • On checkout failure for payload-open PRs, re-query GitHub’s PR API to confirm the current PR state.
  • If the PR is now closed, log a warning and exit successfully (checkout_pr_success: true) instead of failing the job.
  • Adds a dedicated test suite covering the race-condition scenarios and API re-check warning behavior (including HTTP status reporting).
Show a summary per file
File Description
actions/setup/js/checkout_pr_branch.cjs Adds PR state re-check via GitHub API on checkout failure to gracefully handle “PR closed after trigger” races.
actions/setup/js/checkout_pr_branch.test.cjs Adds tests for race-condition handling, API re-check failures, status code logging, and correct API call args.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (2)

actions/setup/js/checkout_pr_branch.cjs:289

  • The new closed-PR race-condition path hard-codes "merged after workflow was triggered" in both the log line and the step summary, but currentPR.state === "closed" also covers PRs that were simply closed (not merged). This makes the warning/summary misleading. Consider either (a) wording it as "closed after workflow was triggered" or (b) checking the merge status from the API response (e.g., merged_at / merged) and tailoring the message accordingly.
      core.warning(`PR state: closed (merged after workflow was triggered)`);
      core.warning(`Checkout failed (expected for closed PR): ${errorMsg}`);

      if (pullRequest.head?.ref) {
        core.warning(`Branch likely deleted: ${pullRequest.head.ref}`);
      }

      core.warning("This is expected behavior when a PR is closed - the branch may have been deleted.");
      core.endGroup();

      // Set output to indicate successful handling of closed PR
      core.setOutput("checkout_pr_success", "true");

      const warningMessage = `## ⚠️ Closed Pull Request

Pull request #${pullRequest.number} was merged after this workflow was triggered. The checkout failed because the branch has been deleted, which is expected behavior.

**This is not an error** - workflows targeting closed PRs will continue normally.`;

actions/setup/js/checkout_pr_branch.cjs:293

  • This change introduces a second block that emits the "Closed PR Checkout Warning", sets checkout_pr_success, and writes a closed-PR summary (the original block handles isClosed from payload; the new one handles isNowClosed from API). The duplicated logic increases the risk of the two paths drifting (e.g., different messages/fields). Consider extracting a small helper to emit the warning + summary so both paths share the same implementation, with only the state text differing.
    if (isNowClosed) {
      core.startGroup("⚠️ Closed PR Checkout Warning");
      core.warning(`Event type: ${eventName}`);
      core.warning(`PR number: ${pullRequest.number}`);
      core.warning(`PR state: closed (merged after workflow was triggered)`);
      core.warning(`Checkout failed (expected for closed PR): ${errorMsg}`);

      if (pullRequest.head?.ref) {
        core.warning(`Branch likely deleted: ${pullRequest.head.ref}`);
      }

      core.warning("This is expected behavior when a PR is closed - the branch may have been deleted.");
      core.endGroup();

      // Set output to indicate successful handling of closed PR
      core.setOutput("checkout_pr_success", "true");

      const warningMessage = `## ⚠️ Closed Pull Request

Pull request #${pullRequest.number} was merged after this workflow was triggered. The checkout failed because the branch has been deleted, which is expected behavior.

**This is not an error** - workflows targeting closed PRs will continue normally.`;

      await core.summary.addRaw(warningMessage).write();
      return;
    }
  • Files reviewed: 2/2 changed files
  • Comments generated: 2

}
} catch (apiError) {
const apiErrorMsg = getErrorMessage(apiError);
const statusCode = /** @type {any} */ apiError?.status;
Copy link

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

JSDoc type assertions in this repo appear to consistently use the parenthesized form (e.g. const err = /** @type {any} */ (error); in actions/setup/js/assign_agent_helpers.cjs:400). Here, the cast is written as /** @type {any} */ apiError?.status, which is inconsistent and can be less reliably understood by TS tooling. Suggest casting apiError first (or using the repo’s existing pattern) before reading .status.

This issue also appears in the following locations of the same file:

  • line 268
  • line 272
Suggested change
const statusCode = /** @type {any} */ apiError?.status;
const typedApiError = /** @type {any} */ (apiError);
const statusCode = typedApiError?.status;

Copilot uses AI. Check for mistakes.
Comment on lines +772 to +787
// Should log as warning, not error
expect(mockCore.startGroup).toHaveBeenCalledWith("⚠️ Closed PR Checkout Warning");
expect(mockCore.warning).toHaveBeenCalledWith("Event type: pull_request");
expect(mockCore.warning).toHaveBeenCalledWith("PR number: 123");
expect(mockCore.warning).toHaveBeenCalledWith("PR state: closed (merged after workflow was triggered)");
expect(mockCore.warning).toHaveBeenCalledWith(expect.stringContaining("couldn't find remote ref"));
expect(mockCore.warning).toHaveBeenCalledWith("Branch likely deleted: feature-branch");
expect(mockCore.warning).toHaveBeenCalledWith("This is expected behavior when a PR is closed - the branch may have been deleted.");

// Should write summary with the "merged after" message
expect(mockCore.summary.addRaw).toHaveBeenCalled();
const summaryCall = mockCore.summary.addRaw.mock.calls[0][0];
expect(summaryCall).toContain("⚠️ Closed Pull Request");
expect(summaryCall).toContain("was merged after this workflow was triggered");
expect(summaryCall).toContain("This is not an error");

Copy link

Copilot AI Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These new assertions hard-code "merged after workflow was triggered" for the closed-PR race condition, but the implementation currently only checks state === "closed" (which also covers unmerged closures). If the production message is adjusted to distinguish closed vs merged (or to use more neutral wording), update these expectations to match the corrected behavior.

Copilot uses AI. Check for mistakes.
@github-actions
Copy link
Copy Markdown
Contributor

🧪 Test Quality Sentinel Report

Test Quality Score: 84/100

Excellent test quality

Metric Value
New/modified tests analyzed 7
✅ Design tests (behavioral contracts) 6 (86%)
⚠️ Implementation tests (low value) 1 (14%)
Tests with error/edge cases 7 (100%)
Duplicate test clusters 0
Test inflation detected YES (156 test lines / 48 production lines = 3.25:1)
🚨 Coding-guideline violations None

Score breakdown: Behavioral (34/40) + Edge cases (30/30) + Duplication (20/20) + Inflation (0/10) = 84/100


Test Classification Details

Test File Classification Issues Detected
should treat checkout failure as warning when PR was merged after workflow triggered checkout_pr_branch.test.cjs ✅ Design Verifies graceful closed-PR handling: no setFailed, correct checkout_pr_success: true, warning group (not error group)
should still fail when PR is still open and checkout fails checkout_pr_branch.test.cjs ✅ Design Verifies non-race failures still propagate: setFailed, output false, no warning group
should still fail when API re-check itself fails checkout_pr_branch.test.cjs ✅ Design Verifies graceful degradation: API failure (429) → fall through to checkout error path
should include HTTP status code in API re-check failure warning checkout_pr_branch.test.cjs ✅ Design Verifies HTTP status appears in diagnostic output (observable triage info)
should omit HTTP status suffix when API error has no status code checkout_pr_branch.test.cjs ✅ Design Verifies status suffix absent for statusless errors; confirms error message included
should call the GitHub API with the correct PR number and repo checkout_pr_branch.test.cjs ⚠️ Implementation Only asserts mock call arguments; no behavioral assertion on output or side-effects
should handle race condition for gh pr checkout path (issue_comment event) checkout_pr_branch.test.cjs ✅ Design Verifies race condition handling generalizes to issue_comment event type

Flagged Tests — Requires Review

⚠️ should call the GitHub API with the correct PR number and repo (actions/setup/js/checkout_pr_branch.test.cjs)

Classification: Implementation test

Issue: The test's sole assertion is expect(mockGithub.rest.pulls.get).toHaveBeenCalledWith({ owner, repo, pull_number }). It verifies internal API call parameters but makes no assertion on the observable outcome — no check on setOutput, setFailed, or whether the closed-PR warning path was taken.

What design invariant does this test enforce? That a specific internal function receives specific argument names (pull_number vs number). This is an internal API binding detail, not a behavioral contract.

What would break if deleted? Only an incorrect API parameter name would go undetected. A behavioral regression (e.g., wrong output state) would still be caught by the other six tests.

Suggested improvement: Add at least one behavioral assertion alongside the parameter check. For example, assert expect(mockCore.setOutput).toHaveBeenCalledWith("checkout_pr_success", "true") to confirm the correct PR was checked and the result was acted on. This transforms the test from "verified the call was made" to "verified the call was made and produced the right outcome".


Test Inflation Note

The test file grew at a 3.25:1 ratio (156 test lines added vs. 48 production lines). This triggers the inflation flag per rubric, though in this case the ratio is largely explained by vitest mock boilerplate (beforeEach/afterEach/describe scaffolding) rather than true test bloat. The 7 tests are all distinct, non-duplicated scenarios. No action required.


Language Support

Tests analyzed:

  • 🐹 Go (*_test.go): 0 tests — no Go test files modified in this PR
  • 🟨 JavaScript (*.test.cjs, *.test.js): 7 tests (vitest) in actions/setup/js/checkout_pr_branch.test.cjs

Verdict

Check passed. 14% of new tests are implementation tests (threshold: 30%). The new test suite provides strong behavioral coverage of the race condition fix across all key scenarios: graceful close handling, continued failure for open PRs, API degradation paths, status-code formatting, and cross-event-type coverage.


📖 Understanding Test Classifications

Design Tests (High Value) verify what the system does:

  • Assert on observable outputs, return values, or state changes
  • Cover error paths and boundary conditions
  • Would catch a behavioral regression if deleted
  • Remain valid even after internal refactoring

Implementation Tests (Low Value) verify how the system does it:

  • Assert on internal function calls (mocking internals)
  • Only test the happy path with typical inputs
  • Break during legitimate refactoring even when behavior is correct
  • Give false assurance: they pass even when the system is wrong

Goal: Shift toward tests that describe the system's behavioral contract — the promises it makes to its users and collaborators.

🧪 Test quality analysis by Test Quality Sentinel · ● 276K ·

Copy link
Copy Markdown
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Test Quality Sentinel: 84/100. Test quality is excellent — 14% of new tests are implementation tests (threshold: 30%). Seven tests comprehensively cover the race condition fix across all key behavioral scenarios. One minor improvement suggested: add a behavioral assertion to the API parameter verification test.

@pelikhan pelikhan merged commit 99f08ce into main Apr 10, 2026
160 of 162 checks passed
@pelikhan pelikhan deleted the copilot/investigate-failure-issue branch April 10, 2026 09:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants