job was stuck by error `Waiting for a runner to pick up this job` for self hosted runners

[Rosyya]

Describe the bug
job sometimes stuck and get logs Waiting for a runner to pick up this job even the self hosted runners are idle, while nothing changed in the workflow yaml file, it will run after minutes or hours

To Reproduce
Steps to reproduce the behavior:

1. create self hosted windows runner
2. run workflow on this runner for several times
3. job stuck by error Waiting for a runner to pick up this job

Expected behavior
runners should be active once get job submitted

Runner Version and Platform

https://github.com/actions/runner/releases/download/v2.320.0/actions-runner-win-x64-2.320.0.zip on Windows

OS of the machine running the runner?
Windows

What's not working?

Waiting for a runner to pick up this job for minutes or hours

Job Log Output

Waiting for a runner to pick up this job for minutes

Runner and Worker's Diagnostic Logs

If applicable, add relevant diagnostic log information. Logs are located in the runner's _diag folder. The runner logs are prefixed with Runner_ and the worker logs are prefixed with Worker_. Each job run correlates to a worker log. All sensitive information should already be masked out, but please double-check before pasting here.

[melyux]

This happens ALL the time. Like more than 20% of the time, we're stuck waiting for this.

[luismcasillas]

It would be helpful to have the option to cancel a job without resorting to the GitHub API's force-cancel feature when the job is in a specific state. Currently, when using concurrency groups, a job that is marked as "in-progress" cannot be canceled if it is waiting for a self-hosted runner to pick it up. The only available option is to use the GitHub API to force-cancel the "in-progress" job.