ci: arm64 self-hosted runner auto-reregister watchdog - #923
Merged
Conversation
added 3 commits
July 27, 2026 08:11
Port runner-watchdog-mac-intel.sh to the Apple Silicon node (Mac-mini-Alonso-227, labels self-hosted,macOS,ARM64,c2pool-mac-arm). arm64 is a required release leg (macOS universal lipo-merge); until now only the Intel node self-healed a server-side deregistration, leaving the arm64 runner as an unguarded single point of failure. Same idempotent, non-destructive design: polls the runners API, and after THRESHOLD consecutive offline checks mints a fresh registration token and re-registers via config.sh --replace (no delete) over SSH. Revert: delete scripts/ci/runner-watchdog-mac-arm.sh + its bridge cron entry.
…nd config.sh --replace Drop test on .227 showed config.sh refuses to reconfigure while the launchd service is still installed (config.sh remove -> "Uninstall service first"; --replace -> "already configured"). Stop alone is not enough. Correct flow: svc.sh stop -> svc.sh uninstall -> config.sh --replace -> svc.sh install -> svc.sh start. Still idempotent, no delete.
A real .227 drop test showed the naive port (mirroring the untested Intel sibling) cannot self-heal: - launchd LaunchAgents must be managed in the user GUI domain over SSH (launchctl bootout/bootstrap gui/$UID); svc.sh install/uninstall alone fail with Input/output error from a non-GUI session. - config.sh --replace does not bypass the local already-configured guard; a real unconfigure (config.sh remove / wipe .runner+.credentials) must precede a fresh configure. Sequence is now: bootout+uninstall -> remove-token unconfigure -> fresh config.sh configure -> svc install + bootstrap+kickstart into gui/$UID. Non-destructive: re-registers under the same name/labels; no orphan delete.
frstrtr
added a commit
that referenced
this pull request
Jul 27, 2026
Ports the three fixes proven by the .227 arm64 drop test (PR #923) onto the Intel node's runner-watchdog: manage the launchd LaunchAgent in the gui/$UID domain, use config.sh remove (not --replace) before a fresh configure, and mint a remove-token to deregister server-side. Drop test on .204 is UNVERIFIED pending SSH access restoration.
frstrtr
added a commit
that referenced
this pull request
Jul 28, 2026
…924) Ports the three fixes proven by the .227 arm64 drop test (PR #923) onto the Intel node's runner-watchdog: manage the launchd LaunchAgent in the gui/$UID domain, use config.sh remove (not --replace) before a fresh configure, and mint a remove-token to deregister server-side. Drop test on .204 is UNVERIFIED pending SSH access restoration.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ports
runner-watchdog-mac-intel.shto the Apple Silicon node Mac-mini-Alonso-227 (labelsself-hosted,macOS,ARM64,c2pool-mac-arm).Why: arm64 is a required release leg (the macOS universal package lipo-merges arm64 + x86_64). Only the Intel node self-healed a server-side deregistration; the arm64 runner had no auto-reregister — an unguarded single point of failure on every release.
Design (identical to the Intel sibling): idempotent, non-destructive. Polls the runners API; after THRESHOLD consecutive offline observations, mints a fresh registration token and re-registers via
config.sh --replace(reuses the same name, no delete) over SSH, then reloads the launchd service. Fail counter clears the moment the runner is seen online.Drop test (real, on .227): service stopped -> runner offline -> watchdog re-registered unattended -> came back online -> picked up a job. Evidence in thread.
Revert: delete
scripts/ci/runner-watchdog-mac-arm.sh+ its bridge cron entry. Never deletes the runner.