Skip to content

Pin downgrade CI to a hosted runner - #126

Merged
ChrisRackauckas merged 1 commit into
SciML:mainfrom
ChrisRackauckas-Claude:agent/pin-downgrade-hosted-runner
Jul 21, 2026
Merged

Pin downgrade CI to a hosted runner#126
ChrisRackauckas merged 1 commit into
SciML:mainfrom
ChrisRackauckas-Claude:agent/pin-downgrade-hosted-runner

Conversation

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member

Ignore this draft until it has been reviewed by @ChrisRackauckas.

Summary

  • pin the downgrade workflow to the reusable workflow's documented ubuntu-24.04 runner input
  • leave the downgrade resolver, dependency floors, and test coverage unchanged

Why

The current main downgrade run (29845314639) completed resolution and build, then lost communication with its self-hosted-4vcpu-8gb runner while the test step was still active. There was no test or resolver failure in the uploaded metadata.

This is runner-specific rather than a repository regression:

  • hosted run 29541435021 passed on commit 7738d76, whose tree e89ed738f45a8865b1fa74fb825af22010c80b74 is byte-identical to current main at a9adb61
  • all 12 recent successful downgrade runs inspected used GitHub-hosted runners; 7 of 8 inspected failed self-hosted runs reported lost runner communication
  • the exact current downgraded suite passed locally, but took 73m43.9s and reached about 6.4 GiB RSS during Gray-Scott

The centralized downgrade workflow exposes os as a public workflow_call input and documents ubuntu-24.04 as the way to force GitHub-hosted execution.

Validation

  • Julia 1.10.11, julia-downgrade-compat@fab1def, exact current dependency graph: Core | 1,288,226 passed / 1,288,226 total | 73m43.9s
  • actionlint .github/workflows/Downgrade.yml
  • Runic 1.7 over every tracked .jl file
  • git diff --check

Investigation process

This was triggered by the fleet downgrade audit. I inspected the exact Actions job and annotations, reproduced the reusable downgrade action on a clean detached checkout of a9adb61, ran the repository's official grouped test harness with the same Julia and Pkg.test options, compared the failed runner with recent hosted and self-hosted history, and used the byte-identical hosted pass as the repository boundary. No tests were skipped, silenced, loosened, or removed.

Co-Authored-By: Chris Rackauckas <accounts@chrisrackauckas.com>
@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Investigation scratchpad (immutable evidence):

  • PR head: 660ae97783fb208168a82442916042e3b23bbf38; base at creation: a9adb61f4c2c8db763fe53988960317dbb1a527c.
  • Failed main job: run 29845314639, job 88684312799, runner ID 1965414, runner self-hosted-4vcpu-8gb-dqgwf-runner-wrl6t. Resolution and build passed; test remained in progress until the runner-lost-communication annotation after 73m06s in the test step.
  • Hosted boundary: run 29541435021, job 87764327053, runner ID 1001010459, runner GitHub Actions 1001010459, passed 1,288,226/1,288,226. Its commit 7738d76f0088102f95bccfdc037d6f06935a04d5 and current main a9adb61f4c2c8db763fe53988960317dbb1a527c have byte-identical tree e89ed738f45a8865b1fa74fb825af22010c80b74 (git diff --exit-code passed).
  • Exact clean-main local reproduction: Julia 1.10.11; julia-downgrade-compat@fab1defb76df9fd672f63c94df73ce131d32e134; mode deps; project .; no-promote Mooncake; exact Actions Pkg.test flags. Result: Core | 1,288,226 / 1,288,226 | 73m43.9s, exit 0.
  • During Gray-Scott, the local worker was observed at about 6.4 GiB RSS / 16.4 GiB VSZ and remained compute-active. This is close to the self-hosted runner class's total 8 GiB memory.
  • Branch validation: actionlint exit 0; Runic 1.7 over all tracked .jl files exit 0; git diff --check exit 0.
  • Raw local test log: /home/crackauc/sandbox/tmp_20260715_055953_80193/audit/logs/finitevolumemethod-a9adb61-exact-downgrade-test-julia-1.10.11.log; SHA-256 e41cab95e588a8f75af283e7986f63d557eb7c4c05b343e0898b15425889cd7b.
  • Conservative CI timer: 2026-07-21 15:32:33 EDT to 17:32:33 EDT, based on a 1h32m05s longest job on the reference PR. CI will not be polled before the timer expires.

The hosted pin is not considered verified until this PR's hosted downgrade job passes.

@ChrisRackauckas
ChrisRackauckas marked this pull request as ready for review July 21, 2026 21:15
@ChrisRackauckas
ChrisRackauckas merged commit e9fdb36 into SciML:main Jul 21, 2026
9 of 10 checks passed
@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Post-timer hosted verification:

  • @ChrisRackauckas marked this ready and merged it as e9fdb36edcdc7a8a528d47b19aaece5f5da2427e.
  • Target downgrade run 29861895572, job 88740432418, ran on runner ID 1001022360 (GitHub Actions 1001022360) with label and image ubuntu-24.04.
  • Result: Core | 1,288,226 / 1,288,226 | 48m36.4s; the complete job passed in 55m55s.
  • The exact hosted job log was archived locally as audit/logs/finitevolumemethod-pr126-hosted-downgrade-success.log, SHA-256 b591bea145e90cc741ddfceee34e9e97b8aa1b1efc81c7901bdc76200a30b13f.

All three Core matrix legs also passed. The Julia 1 QA leg failed independently of this downgrade-only diff and is being handled as a separate clean-main investigation; it does not alter the hosted downgrade verification above.

@ChrisRackauckas-Claude

Copy link
Copy Markdown
Member Author

Post-merge default-head verification:

  • Current main head: e9fdb36edcdc7a8a528d47b19aaece5f5da2427e.
  • Push run 29869241196, job 88765029692, passed on runner ID 1001022457 (GitHub Actions 1001022457) with label and image ubuntu-24.04.
  • Result: Core | 1,288,226 / 1,288,226 | 45m39.9s; complete job duration 52m34s.
  • Raw hosted log: audit/logs/finitevolumemethod-main-e9f-downgrade-success.log, SHA-256 88a3700f51d39823f5f2bc5e9cdf47f10a66e118782c662265a6919a3edfbeec.
  • Job metadata: audit/logs/finitevolumemethod-main-e9f-downgrade-run-29869241196-jobs.json, SHA-256 2895db2755b1e797ba9c74ca722404526669a64b69552a7b935841148700216f.

This verifies the hosted downgrade pin both on the PR head and on the exact current default branch head.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants