Provide immediate cancel-in-progress concurrency

[Naros]

🏷️ Discussion Type

Product Feedback

💬 Feature/Topic Area

ARC (Actions Runner Controller)

Discussion Details

Our repository's push workflow has concurrency set up as follows:

concurrency:
  group: ${{ github.workflow }}-${{ github.ref_name }}
  cancel-in-progress: true

The problem we face is that our workflow includes jobs that can take 60 to 150 minutes to complete, as they run a battery of integration tests across multiple databases. We had hoped that using cancel-in-progress would trigger an immediate termination of any active steps and jobs within a matter of minutes, allowing the next queued workflow to start.

What we have found is that while this prevents the in-progress workflow from queuing up any further workflow jobs or steps within the current job, any active job steps run until they complete, rather than being terminated/killed. Given the duration of these steps, as I mentioned, this caused some pushes to wait 1h-2h before they ever start. For a CI pipeline that already takes approximately 5h to complete, this just exacerbates things.

Is there any plan or way to explicitly trigger an immediate cancellation of a workflow without having to wait on any existing active job steps to complete first?

[github-actions[bot]]

💬 Your Product Feedback Has Been Submitted 🎉

Thank you for taking the time to share your insights with us! Your feedback is invaluable as we build a better GitHub experience for all our users.

Here's what you can expect moving forward ⏩

• Your input will be carefully reviewed and cataloged by members of our product teams. 
  • Due to the high volume of submissions, we may not always be able to provide individual responses.
  • Rest assured, your feedback will help chart our course for product improvements.
• Other users may engage with your post, sharing their own perspectives or experiences. 
• GitHub staff may reach out for further clarification or insight. 
  • We may 'Answer' your discussion if there is a current solution, workaround, or roadmap/changelog post related to the feedback.

Where to look to see what's shipping 👀

• Read the Changelog for real-time updates on the latest GitHub features, enhancements, and calls for feedback.
• Explore our Product Roadmap, which details upcoming major releases and initiatives.

What you can do in the meantime 💻

• Upvote and comment on other user feedback Discussions that resonate with you.
• Add more information at any point! Useful details include: use cases, relevant labels, desired outcomes, and any accompanying screenshots.

As a member of the GitHub community, your participation is essential. While we can't promise that every suggestion will be implemented, we want to emphasize that your feedback is instrumental in guiding our decisions and priorities.

Thank you once again for your contribution to making GitHub even better! We're grateful for your ongoing support and collaboration in shaping the future of our platform. ⭐

[xusnitdinov]

@Naros yeah, cancel-in-progress: true is more "please stop scheduling new work" than "yank the power cord on a 2-hour job right now." Long-running steps (especially on self-hosted / ARC) often keep chewing until the current process exits or the runner tears down.

Workarounds that help today while the feature request cooks:

1. Split the monster job: short cancelable prep + long integration jobs in separate workflows / concurrency groups so a cancel doesn't strand everything.
2. Add cooperative cancellation inside the long step (poll cancelled() / check the run status and exit early).
3. For ARC: look at runner pod cleanup / job TTL settings so cancelled jobs don't sit around holding capacity.
4. Use a tighter concurrency group for the long tests only, so a docs-only push doesn't try to cancel a 150-minute suite.

Your ask is still valid though: "cancel-in-progress should mean minutes, not wait-for-current-step." That distinction belongs in the docs if GitHub can't make cancellation hard-stop everywhere.

Curious: are these GitHub-hosted runners or ARC self-hosted? Behavior differs a lot there.

[initial-d]

I would distinguish two layers here:

1. GitHub marking the old run/job as cancelled because a newer run entered the same concurrency group.
2. The runner actually terminating the process tree that is currently executing your long integration step.

The concurrency docs describe the scheduling/cancellation policy:

https://docs.github.com/actions/writing-workflows/choosing-what-your-workflow-does/control-the-concurrency-of-workflows-and-jobs

They explain that cancel-in-progress: true cancels an in-progress run/job in the same group, but in practice the speed of resource release still depends on how the runner and the active process respond to cancellation.

For GitHub-hosted runners, cancellation is usually reasonably quick because GitHub owns the VM lifecycle. For self-hosted/ARC runners, a long-running child process can keep doing work unless it handles termination or the pod/runner is forcefully cleaned up.

A few mitigations that have helped in long integration suites:

1. Put concurrency at the job level for the expensive matrix, not only the workflow level.

   That prevents unrelated cheap jobs from being tied to the same cancellation behavior.

   jobs:
     integration:
       concurrency:
         group: integration-${{ github.ref_name }}-${{ matrix.database }}
         cancel-in-progress: true

2. Add cooperative cancellation inside the long-running test harness.

   For example, wrap the integration command in a small script that periodically checks the GitHub run status through the API and exits if the run is no longer active. This is especially useful if the real work is one giant shell command, Docker Compose run, Gradle task, or database test runner.

3. Make the long step respond to SIGTERM and clean up its child processes.

   If the step launches containers, background jobs, nested shells, or test workers, make sure the wrapper traps termination and stops them:

   cleanup() {
     docker compose down -v || true
     jobs -p | xargs -r kill || true
   }
   trap cleanup TERM INT EXIT
   
   ./run-long-integration-suite.sh &
   wait $!

4. For ARC/self-hosted, check runner pod termination behavior.

   If the pod gets a termination signal but the test process ignores it, Kubernetes may wait for the termination grace period. A long grace period, stuck child process, or cleanup finalizer can look like cancel-in-progress did not work. Review the pod terminationGracePeriodSeconds, runner controller logs, and whether cancelled runner pods actually terminate.

5. Split “should this run?” from “run the 150-minute suite.”

   Use a quick gate job to decide whether the expensive suite is needed, then start the expensive job only when the commit still matters. That avoids burning 1-2 hours on commits that are obsolete before they reach the heavy tests.

Your product request is still valid: if GitHub cannot guarantee hard termination, the docs/UI should probably distinguish “run marked cancelled” from “active runner process has actually stopped.” For ARC users, having a first-class “force terminate current runner pod for cancelled job” behavior would be much closer to what people expect from cancel-in-progress: true.