Self-hosted runner jobs repeatedly canceled server-side ~30 min in, healthy runner, no user action

[paomian]

🏷️ Discussion Type

Question

💬 Feature/Topic Area

Actions Runner

Discussion Details

Summary
On my public fork paomian/greptimedb, workflow_dispatch runs of the "Query Regression" workflow are repeatedly canceled by the GitHub service roughly 27–32 minutes into the main job, while the self-hosted runner is healthy and stays online. I never cancel these runs, and nothing in my account's security log shows a cancel event.

Runs affected (all workflow_dispatch on branch perf-aliyun-ecs):

https://github.com/paomian/greptimedb/actions/runs/32568232823 — job started 2026-08-22T10:41:49Z, canceled at 11:08:43Z
Evidence the runner/machine was healthy at cancellation time:

The runner's serial console shows the job receiving SIGTERM with no preceding local failure, then reporting "Job query-regression completed with result: Canceled". No OOM, no crash, no network error.
In run 32568232823 the "Build base greptime" step completed successfully in 24 minutes; the next step was canceled 2.5 minutes in. The "Complete job" step ran fine afterwards, i.e. the runner was still connected and reporting.
Steps with if: always() were skipped, indicating a service-side job cancellation rather than a step failure.
concurrency in this workflow uses cancel-in-progress: false, so no queued run cancels it. No automation in the repo calls the cancel API. My personal access token is only used by the provisioning scripts (runner registration), never to cancel runs.
What I need
Please check the backend logs for these run ids and tell me what triggered the cancellation (which actor or which system component), and whether there is any platform-side limit or heuristic that cancels long-running jobs on self-hosted runners for free personal accounts. Three runs have now died at nearly the same offset (~30 minutes) after job start, which looks systematic rather than incidental.

[Pusri27]

🏷️ Discussion Type

Question

💬 Feature/Topic Area

Actions Runner

Discussion Details

Summary On my public fork paomian/greptimedb, workflow_dispatch runs of the "Query Regression" workflow are repeatedly canceled by the GitHub service roughly 27–32 minutes into the main job, while the self-hosted runner is healthy and stays online. I never cancel these runs, and nothing in my account's security log shows a cancel event.

Runs affected (all workflow_dispatch on branch perf-aliyun-ecs):

https://github.com/paomian/greptimedb/actions/runs/32568232823 — job started 2026-08-22T10:41:49Z, canceled at 11:08:43Z (add the other run ids here) Evidence the runner/machine was healthy at cancellation time:

The runner's serial console shows the job receiving SIGTERM with no preceding local failure, then reporting "Job query-regression completed with result: Canceled". No OOM, no crash, no network error. In run 32568232823 the "Build base greptime" step completed successfully in 24 minutes; the next step was canceled 2.5 minutes in. The "Complete job" step ran fine afterwards, i.e. the runner was still connected and reporting. Steps with if: always() were skipped, indicating a service-side job cancellation rather than a step failure. concurrency in this workflow uses cancel-in-progress: false, so no queued run cancels it. No automation in the repo calls the cancel API. My personal access token is only used by the provisioning scripts (runner registration), never to cancel runs. What I need Please check the backend logs for these run ids and tell me what triggered the cancellation (which actor or which system component), and whether there is any platform-side limit or heuristic that cancels long-running jobs on self-hosted runners for free personal accounts. Three runs have now died at nearly the same offset (~30 minutes) after job start, which looks systematic rather than incidental.

Hey @paomian,

Seeing repeated cancellations right around the ~30-minute mark without any local OOM or crash definitely feels like a server-side heartbeat drop or an implicit timeout tripping up somewhere.

A few things worth looking into:

• Runner Diagnostic Logs (_diag folder): The serial console only shows the graceful shutdown (SIGTERM), but the runner's internal log trace will show why the job listener gave up. Check <runner_dir>/_diag/Runner_*.log and Worker_*.log around the 27–32 min mark. Look for lines mentioning MessageListener, SessionAbandoned, or any HTTP 401/408 drops.
• Transient Network / Session Drop: Self-hosted runners maintain a long-polling session with GitHub. If there's an outbound network hiccup, firewall state drop, or proxy timeout on the ECS instance that cuts the connection to pipelines.actions.githubusercontent.com, GitHub's backend flags the runner as lost and issues a cancellation signal.
• Set an Explicit timeout-minutes: Even though self-hosted jobs default to a much longer execution window, try adding timeout-minutes: 120 directly under the job in your YAML just to rule out any weird upstream default calculation.

Just a heads-up that this forum is mainly peer-to-peer, so nobody here has access to GitHub's internal backend trace logs to inspect what actor/service triggered the cancel on run 32568232823.

If the _diag logs confirm your machine had zero network drops to GitHub's endpoints, your best bet is opening a quick ticket at [GitHub Support](https://support.github.com/) with that run ID and your zipped _diag folder so an engineer can check the internal trace.

[justin24er]

This really does smell systematic given the consistent ~27-32 min mark across multiple runs🤔, that's too tight a pattern to be coincidental infra flakiness.

A few things worth checking/mentioning before this gets escalated:

• Self-hosted runners on public repo forks are subject to extra scrutiny, GitHub has anti-abuse heuristics specifically for self-hosted runners on public repos (since malicious forks could otherwise run arbitrary code on someone's hardware via PRs). One known behavior: GitHub may auto-cancel jobs on self-hosted runners for public repos when it detects certain risk signals, this exists precisely because self-hosted runners on public repos are a known attack vector.

• Check if pull_request triggers are enabled alongside workflow_dispatch on this workflow, if a fork PR came in or a stale approval/label state exists concurrently, GitHub's fork-safety system can cancel jobs even on unrelated dispatch runs.

• Look in the repo's Security tab / Actions security log at the org level (not just your personal audit log), cancellations from platform-level actors sometimes don't show up in the standard user security log at all.

Since your own audit log shows nothing and no automation/token triggered it, this is squarely a "GitHub-side actor" question that only staff can actually answer by pulling backend logs for that run ID, so flag clearly in your post (as you did) that you specifically need someone to check the Actions run internals, not general troubleshooting steps, that'll help it get routed to the right team rather than getting a generic self-service reply.