GitHub-hosted Ubuntu runner unexpectedly shuts down during npm install (exit 143, always() step skipped)

[thtna]

🏷️ Discussion Type

Question

💬 Feature/Topic Area

Actions Runner

Discussion Details

Hello GitHub Community,

I'm investigating an unexpected shutdown of a GitHub-hosted Ubuntu runner during an active GitHub Actions job.

Environment

• Runner: GitHub-hosted ubuntu-24.04
• Runner version: 2.338.0
• Runner image version: 20261012.334.1
• Workflow trigger: workflow_dispatch
• Job timeout: 25 minutes
• Step timeout: 23 minutes
• Concurrency: cancel-in-progress: false

What happened

The workflow successfully reached a Docker container execution stage and started npm ci --offline.

Approximately 10 seconds after npm started, the job reported:

The runner has received a shutdown signal.
This can happen when the runner service is stopped,
or a manually started runner is canceled.

Process completed with exit code 143.

The exit code 143 belongs to the host workflow step. No npm exit code was captured.

The last available container heartbeat showed no OOM kill, but the final container state could not be verified.

The job concluded as failure, not cancelled.

A subsequent artifact-upload step configured with if: always() was skipped.

What I investigated

I reviewed GitHub Actions Runner v2.338.0 source code and found that runner shutdown can bypass step-condition evaluation, explaining why an always() step may be skipped.

However, this does not identify what initiated the shutdown.

The job had not reached its configured timeouts, and I found no evidence of a competing workflow cancellation.

Questions

1. What conditions can cause a GitHub-hosted runner to receive a shutdown signal while a job is actively running?
2. Is there a way to distinguish user cancellation, control-plane termination, runner service shutdown, and infrastructure interruption?
3. Can billing enforcement or ephemeral-runner lifecycle events produce this behavior?
4. What diagnostic information is available to users when the runner shuts down before artifact upload?
5. Are there recommended practices for preserving minimal diagnostic evidence during an unexpected runner shutdown?

I have not retried the workflow because I would first like to understand the possible causes and the appropriate diagnostic approach.

Any guidance from the community would be appreciated. Thank you!

[Sagargupta16]

The message narrows it down. "The runner has received a shutdown signal" is written only when the runner's own shutdown token fires with reason UserCancelled (JobRunner.cs L133). That token is set when the Runner.Listener process gets SIGTERM or Ctrl-C (Runner.cs L350-L390) or when the Worker process itself is unloaded (Worker.cs L221). A cancel from the UI or API, a timeout-minutes expiry or a concurrency cancel reaches the worker as a plain CancelRequest instead, and the job ends as cancelled. StepsRunner.cs makes the split: on cancellation the result is Failed if the shutdown token fired, otherwise Canceled, and with the token set it skips condition evaluation ("Skip evaluate condition on runner shutdown"), so if: always() is never checked. An OS shutdown gives a third message, "Operating system is shutting down for computer ...". So the job was not cancelled or timed out: the runner process on the VM was told to stop from outside your job. The source does not say by whom, and on a hosted VM only GitHub's side can.

Exit 143 is the step, not npm: on cancel the runner sends your step SIGINT, waits 7.5 s, then SIGTERM (ProcessInvoker.cs L32-L33), and 128+15 is bash reporting the SIGTERM.

Two things you can do. Set the repository variable ACTIONS_RUNNER_DEBUG to true (Enabling debug logging): the downloaded log archive then has a runner-diagnostic-logs folder with Runner_*.log and Worker_*.log, where the shutdown appears as "Runner will be shutdown for UserCancelled". That upload runs during job finalization inside the 30 s unload window, so a hard teardown can still lose it. Evidence that survives regardless: step stdout is streamed to GitHub every 250 to 500 ms (JobServerQueue.cs), while files and artifacts need a later step you will not get. A background loop printing free -m, df -h and docker stats --no-stream into the step log every few seconds is the only record that outlives the VM.

On billing: the docs describe blocking usage before a job starts; I found nothing about stopping a running job and could not confirm it either way. Retry the run. If it dies again on the same step, the run URL plus that streamed log is what Support needs.

[NexzaDev]

Sagargupta16's analysis of the runner source explains the "why" well. Here is a bit more on your remaining questions.

What can trigger a shutdown signal mid-job?

As noted above, that message means the runner process itself was told to stop, not that the job was cancelled or timed out. On a GitHub-hosted VM that has to come from the platform side (host maintenance, an infrastructure interruption, or the service tearing down the runner). Users can't see which one it was. OOM is an unlikely cause, since the kernel OOM killer sends SIGKILL, which normally shows up as exit 137 or a "lost communication" error rather than a graceful shutdown message.

Telling the cases apart

From the user side you can mostly go by the message and the final conclusion:

• Cancelled via the UI/API, timeout-minutes, or concurrency: the job ends as cancelled.
• "The runner has received a shutdown signal": the job ends as failed, and the runner process was stopped externally.
• "Operating system is shutting down for computer ...": the OS itself shut down.
• "The hosted runner lost communication with the server": usually resource starvation or a network loss.

Distinguishing control-plane termination from an infrastructure interruption isn't possible from your logs. Only GitHub Support can see that.

Billing

I can't confirm this either way. The docs describe billing limits blocking jobs from starting, but I haven't found anything saying a running job gets stopped. It is worth checking your Billing page for spending-limit or quota alerts around that timestamp.

Preserving evidence

Anything that depends on a later step (artifact upload, always()) is lost in this scenario, so it helps to write diagnostics to the step log, which is streamed as the job runs. For example, start this before npm ci:

( while true; do
    echo "=== $(date -u +%T) ==="
    free -m | sed -n 2p
    df -h / | tail -1
    docker stats --no-stream 2>/dev/null
    sleep 5
  done ) &

trap 'echo "SIGTERM at $(date -u +%T)"; uptime; ps aux --sort=-%mem | head -5; sudo dmesg | tail -20' TERM INT

The trap may not finish before teardown, but even a timestamp showing when the signal arrived is useful. I would also enable ACTIONS_RUNNER_DEBUG and ACTIONS_STEP_DEBUG, and check githubstatus.com for incidents around that time.

Next step

Re-run with the monitoring in place. If it fails again at the same step, open a GitHub Support ticket with the run URL, job ID, runner name, and UTC timestamps. A single occurrence is most likely platform-side, but a failure that repeats at npm ci would point to memory or disk pressure inside the container.

Hope this helps.

[thtna]

Thank you, NexzaDev. This is a very helpful explanation.

Our original run concluded as failure, not cancelled. The shutdown occurred roughly 10 seconds after npm ci --offline started inside Docker. The exit code 143 belongs to the host step; npm's own exit code was never captured.

The last available container heartbeat showed oom_kill=0, but we cannot verify the container's final state after that sample.

I agree that better diagnostics would help. Before considering a rerun, we plan to inspect the existing workflow log archive for any runner or worker diagnostic records.

One question: Is there any documented, user-accessible event or metadata field that can identify who initiated the runner shutdown, without requiring GitHub's internal infrastructure logs?

We will keep the distinction between a confirmed runner shutdown and its still-unknown initiator.

Thank you again for your guidance!