Skip to content

[WIP] Investigate CI failure due to fuzz job cancellation#18344

Closed
Copilot wants to merge 1 commit intomainfrom
copilot/fix-fuzz-job-cancellation-issue
Closed

[WIP] Investigate CI failure due to fuzz job cancellation#18344
Copilot wants to merge 1 commit intomainfrom
copilot/fix-fuzz-job-cancellation-issue

Conversation

Copy link
Contributor

Copilot AI commented Feb 25, 2026

Thanks for assigning this issue to me. I'm starting to work on it and will keep this PR's description up to date as I form a plan and make progress.

Original prompt

This section details on the original issue you should resolve

<issue_title>[CI Failure Doctor] CI Failure Doctor - Run 22396700002: fuzz job cancelled by concurrency</issue_title>
<issue_description>### CI Failure Investigation - Run github/gh-aw#37830

Summary

The fuzz job for run 37830 (commit 6867b8113f22940a4c3ebfffdfb1f99830ea1523) was terminated midway through the sequential fuzz targets because GitHub Actions’ cancel-in-progress concurrency guard kicked in when a newer CI workflow (run 37832) started on the same branch. The job never reported any test errors—only a cancellation message—so no defects were introduced by the commit itself.

Failure Details

  • Run: 22396700002
  • Commit: 6867b8113f22940a4c3ebfffdfb1f99830ea1523
  • Trigger: push (main)

Root Cause Analysis

The fuzz job defines a concurrency group named ci-$\{\{ github.ref }}-fuzz with cancel-in-progress: true, so when run 37832 (sha 38ab34435fb7c33057d7a1935f21e490ba350e82) spun up at 2026-02-25T12:26:52Z, GitHub Actions automatically cancelled the earlier run (37830) while it was still executing FuzzExpandLabelTriggerShorthand. The job logs end with ##[error]The operation was canceled. at 2026-02-25T12:28:07Z, and no fuzz test reported an unexpected failure prior to cancellation.

Failed Jobs and Errors

  • fuzz (ubuntu-latest, job Run fuzz tests step): ##[error]The operation was canceled. while executing FuzzExpandLabelTriggerShorthand after completing 17 of the 24 queued fuzz targets. No assertions or module downloads failed.
Investigation Findings
  • Run 22396700002 (run number 37830) spent ~4½ minutes running sequential fuzz targets before being cancelled; the log shows every earlier target completing successfully and the cancellation happening immediately after the 17th fuzz
  • The concurrency definition in .github/workflows/ci.yml intentionally prevents overlapping fuzz jobs by cancelling prior runs, so the cancellation is the expected behavior when another push lands on main
  • Subsequent runs confirm the pattern: run 37832 (sha 38ab3443…) started at 12:26:52Z, run 37833 was cancelled shortly afterwards, and run 37834 is currently in progress, which is why the older run 37830 could not finish

Recommended Actions

  • Re-run the CI workflow for commit 6867b8113f22940a4c3ebfffdfb1f99830ea1523 if the fuzz results from that specific commit are still needed; the job had not failed, it was merely cancelled by the concurrency guard.
  • Only adjust the concurrency guard if you want to allow multiple overlapping fuzz runs on main; otherwise the existing cancel-in-progress setting is doing what it should and no code change is required.

Prevention Strategies

Because the fuzz job runs a long series of sequential targets, it will always be cancelled when a newer push lands while it is still running; the best way to avoid these log entries is to let the latest run finish (no human action needed) or schedule the fuzz suite behind a manual gate. Re-running the workflow on the same commit will produce the full results.

AI Team Self-Improvement

Before concluding that the fuzz job itself is broken, check for additional workflow runs in the same concurrency group and verify whether a ##[error]The operation was canceled. message lines up with cancel-in-progress being triggered; that will prevent misclassifying expected cancellations as regressions.

Historical Context
  • The CI workflow is currently on run 37834 for 92baa35c… and run 37833 was itself cancelled, showing that the queue is active and newer pushes displace older builds
  • The cancellation log for run 37830 is consistent with previously recorded concurrency cancellations (e.g., run 37832 also terminated early, run 37829 failed for a different reason)

🩺 Diagnosis provided by CI Failure Doctor

To install this workflow, run gh aw add githubnext/agentics/workflows/ci-doctor.md@ea350161ad5dcc9624cf510f134c6a9e39a6f94d. View source at https://github.com/githubnext/agentics/tree/ea350161ad5dcc9624cf510f134c6a9e39a6f94d/workflows/ci-doctor.md.

  • expires on Feb 26, 2026, 12:35 PM UTC

Comments on the Issue (you are @copilot in this section)


🔒 GitHub Advanced Security automatically protects Copilot coding agent pull requests. You can protect all pull requests by enabling Advanced Security for your repositories. Learn more about Advanced Security.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CI Failure Doctor] CI Failure Doctor - Run 22396700002: fuzz job cancelled by concurrency

2 participants