Skip to content

ci: add per-job timeout-minutes so a wedged runner fails fast - #71

Merged
aperim-agent merged 1 commit into
mainfrom
ci/job-timeouts
Jun 10, 2026
Merged

ci: add per-job timeout-minutes so a wedged runner fails fast#71
aperim-agent merged 1 commit into
mainfrom
ci/job-timeouts

Conversation

@aperim-agent

Copy link
Copy Markdown
Collaborator

The gap

No job in .github/workflows/ set timeout-minutes. When a GitHub-hosted runner wedges (which has happened repeatedly this week), the job sits in_progress for GitHub's 6-hour default before it's killed — blocking merges and forcing manual cancels.

The fix

Add a job-level timeout-minutes to every job — 15 jobs across all 5 workflows. Each value is sized comfortably above that job's worst-case (cold-cache) runtime so a normal run never trips it, but well under 6h so a hung/wedged runner fails fast and the PR can simply be re-run. Job-level is the right granularity: it caps the whole job, including a runner that wedges before any step starts. When unsure, erred higher — a too-low timeout that kills real work is worse than a slightly-too-high one.

Values chosen + rationale

ci.yml (9 jobs)

Job Worst-case observed timeout
changes (classify) ~30s (checkout + paths-filter) 10
lint (fmt + clippy) ~1-2min warm, higher cold 20
test (check + test, ubuntu + macOS) ubuntu ~7min, macOS ~15min — the long pole; generous headroom for cold macOS cache 40
feature-clippy (per crate/feature leg) ~1min each, cold per-feature cache higher 20
asyncapi-validate ~1min (node setup + cli install + validate) 15
deny (cargo-deny) ~30s run + cold toolchain/binary install 20
inclusive-language seconds 10
docs-sanity seconds 10
web (npm install + Vite build) ~1-5min, cold npm cache higher 20

docker.yml (2 jobs)

Job Rationale timeout
build Full multi-stage Docker build (cargo release build of the whole workspace inside the image) + push by digest, on native amd64/arm64 runners — heaviest leg in the repo; cold cache + native arm64 run long 90
merge No build — imagetools manifest create + provenance/SBOM verify + attest + cosign sign; network/registry-bound 30

ffmpeg-base.yml (2 jobs)

Job Rationale timeout
build ~10-min FFmpeg-from-source compile per leg (per the workflow header) + push by digest, native runners; cold/arm64 run longer 90
merge Same shape as docker merge (manifest + attest + sign) 30

release.yml (1 job)

Job Rationale timeout
build (per target) FFmpeg install + --release --locked cargo build, plus macOS codesign + notarytool --wait which can block on Apple's notarization service 60

release-plz.yml (1 job)

Job Rationale timeout
release-pr Full-history checkout + toolchain + release-plz changelog/version compute + PR open/update (~2-5min) 20

No logic changed

Purely additive: 50 insertions, 0 deletions — only the timeout-minutes key (plus a one-line rationale comment) is added per job. No job's steps, logic, or triggers were touched. No Rust/web code touched, so this will not conflict with parallel work.

Verification

  • actionlint (v1.7.7) run on every workflow file: passes. The only two notes are pre-existing SC2153 info-level shellcheck false positives (DIGESTS env var) in the unmodified merge-job scripts — confirmed identical on pristine origin/main, not introduced here.
  • Job-count vs timeout-count: ci.yml 9/9, docker.yml 2/2, ffmpeg-base.yml 2/2, release-plz.yml 1/1, release.yml 1/1 — every job has exactly one timeout.

🤖 Generated with Claude Code

No job in .github/workflows/ set timeout-minutes, so a wedged GitHub-hosted
runner sat "in_progress" for GitHub's 6h default — blocking merges and forcing
manual cancels. Add a job-level timeout-minutes to every job (15 jobs across 5
workflows), sized comfortably above each job's worst-case (cold-cache) runtime
so normal runs never trip, but well under 6h so a hung runner fails fast and the
PR can be re-run.

Values: ci.yml changes/inclusive-language/docs-sanity 10; lint/feature-clippy/
deny/web 20; asyncapi-validate 15; test (macOS is the long pole) 40. docker.yml
+ ffmpeg-base.yml build 90 (full image/source builds on native runners), merge
30. release.yml build 60 (release build + macOS notarization wait). release-plz
20.

Pure CI-reliability change: only the timeout-minutes key is added — no job
steps, logic, or triggers touched. actionlint passes (the two pre-existing
SC2153 info notes are in unmodified merge-job scripts and are false positives).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@aperim-agent
aperim-agent merged commit 274965c into main Jun 10, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant