ci: cross-OS solx-tester suite on free macOS/Windows runners (ci:cross-os label + weekly cron) - #553
Draft
nebasuke wants to merge 3 commits into
Draft
ci: cross-OS solx-tester suite on free macOS/Windows runners (ci:cross-os label + weekly cron)#553nebasuke wants to merge 3 commits into
nebasuke wants to merge 3 commits into
Conversation
nebasuke
force-pushed
the
cross-os-tester
branch
2 times, most recently
from
July 16, 2026 17:35
61986af to
9ced659
Compare
Distills the mac debug harness (PR #546) and the Windows spike into a reusable composite action: build solx/solx-tester, smoke test, suite run under a parametrized watchdog with per-OS hang diagnostics (macOS: sample of tester + top-level solx parents + pooled workers, crash-report collection; Windows: tasklist snapshot), named signals, and stage outcomes aggregated at the end so diagnostics always run. The workflow runs the full suite on the free macos-15 and windows-2025 runners, with the two Windows prerequisites proven by the spike: drop the bundled static libstdc++.a (solx#550) and keep solx-solidity/test through disk cleanup. Trigger is temporarily push-to-branch for verification; Stage 3 switches it to a ci:cross-os label + workflow_dispatch + weekly cron on main.
Verification run 29429860239 is green on both legs (macos-15 full suite in ~22 min job time, windows-2025 in ~37 min, free runners), so the temporary push trigger is replaced with the real ones: the ci:cross-os PR label for on-demand validation of platform-sensitive changes, workflow_dispatch, and a Monday-morning cron on main as a drift net. Advisory only — label-gated required checks can hang the merge queue. Also names the matrix job (the include entries leaked into the display name) and documents the label in CLAUDE.md.
Unblocked by the rustc 1.97.1 bump (#556) fixing the enum-niche miscompile that killed solx-tester on Intel macs (#547). Full suite, same triggers as the other legs; no post-1.97.1 Intel timing exists, so the watchdog (110 min) and job timeout (240 min) are a probe budget to be tightened once real runs establish a baseline. test.yaml's Intel leg already runs on this runner, so the finished-install LLVM/solc caches on main serve it.
nebasuke
force-pushed
the
cross-os-tester
branch
from
July 21, 2026 08:16
9ced659 to
3eead44
Compare
Coverage Summary
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A label-gated + weekly-cron workflow running the full
solx-testersuite on free macOS (arm64 + Intel) and Windows runners, plus therun-tester-suitecomposite action it is built from. The workflow consumes thebuild-toolchaincomposite (landed separately in #581) for its standard-config LLVM/solc prologue, so it adds no toolchain-build steps of its own.ci:cross-os(on-demand, for platform-sensitive changes like process/IPC work),workflow_dispatch, and a Monday 06:00 UTC cron on main as a drift net. Advisory only — must never become a required check.macos-15,macos-15-intel, andwindows-2025are free for public repos; LLVM/solc come from the finished-install caches saved on main, so nothing heavier than the Rust workspace compiles.Depends on
#581 (merged) — extracted the standard-config toolchain prologue (submodules → SFW → msys → macOS-disk → LLVM → solc) into the shared
build-toolchaincomposite. This workflow is its sixth consumer; the only change to the composite here is addingcross-os-tests.yamlto its consumer-list description. #581 also folded in #550's fix option 2, so the bundled staticlibstdc++.adrop now happens insidebuild-toolchainfor every standard-config Windows build — this workflow inherits the working shared-libstdc++link with no step of its own.Why
One week of evidence that this layer catches what Linux cannot:
libstdc++drop (now inbuild-toolchain) and the corpus-dir retention below were in place.How it works
run-tester-suite(composite action): builds solx + solx-tester (symbols kept), smoke test, then the suite under a parametrized watchdog. A run that neither crashes nor finishes gets diagnosed before the kill — on macOS,samplestacks of the tester, the top-level solx parents, and the pooled workers, plus new crash reports fromDiagnosticReports; on Windows, atasklistsnapshot. Signal deaths are reported by name (killed by signal 11 (SIGSEGV)) instead of thesolx-devwrapper's "subprocess failed without exit code". Stage outcomes are aggregated at the end so diagnostics always run.Windows specifics:
libstdc++.adrop (Windows: fresh llvm-sys builds cannot link against the bundled static libstdc++.a under MSYS2 GCC 16 — CI is one cache-fingerprint change from breaking #550) is inherited frombuild-toolchain, applied to all standard-config Windows test builds — not a step this workflow adds;solx-solidity/test/through the disk cleanup (the Ethereum corpus is read from it at run time) — this workflow's ownFree disk space (Windows)step.Verification
Run 29429860239:
macos-15andwindows-2025legs green, full suites — ~22 min and ~37 min job time respectively. That run predates the rebase ontobuild-toolchain, but the composite encodes byte-identical LLVM/solc params, so the tester results carry over; a freshci:cross-osrun on the rebased branch confirms the composite path.The
macos-15-intelleg has no post-1.97.1 timing yet (pre-fix runs exceeded 75 min without finishing), so its watchdog (110 min) and job timeout (240 min) are a probe budget rather than the ~2x margin the other legs get. First runs establish the baseline; then tighten, or scope down to--path tests/solidityif the full suite can't finish on the free Intel minis.Follow-ups (not in this PR)
libstdc++.ainto the LLVM tree in solx-dev and make release packaging link it statically on its own. That's a solx-dev + release.yaml change whose verification surface is a release build, so it stays out of CI-only work like this. When it lands,build-toolchain's drop step gets deleted — and its plainrmfails loudly if the bundling disappears first, so the workaround can't linger silently.mac-tester-debug*/win-tester-spikebranches and ci: debug solx-tester on macOS runners (main baseline for #524 mac validation) #546 get retired once this lands.