Skip to content

Conversation

@sanity
Copy link
Collaborator

@sanity sanity commented Nov 30, 2025

Problem

Hit the $300/month GitHub Actions budget. Analysis showed expensive jobs running on freenet-default-runner (larger GitHub-hosted runner).

Solution

Set up a self-hosted runner on nova (128GB RAM, fast network) and switched CI to use it.

Changes

  1. Self-hosted runner - Configured nova-runner with labels self-hosted, Linux, X64, nova

    • Systemd service auto-starts on boot
    • Uses existing mold linker and sccache for fast builds
    • 128GB RAM handles six-peer tests easily
  2. Native concurrency cancellation - Added concurrency groups to auto-cancel in-progress runs when new commits push

  3. Reduced timeout - six_peer_regression timeout reduced from 120m → 30m

  4. Skip cross-compile on PRs - Release binaries only need to build on main/tags

Cost Impact

  • Before: ~$300/month on GitHub-hosted runners
  • After: $0 (self-hosted runner is free)

Testing

This PR will test itself on the new nova runner.

[AI-assisted - Claude]

- Add concurrency groups to cancel in-progress runs on new commits
- Switch Test, Clippy, six-peer tests from freenet-default-runner to ubuntu-latest
- Remove redundant cancel-workflow-action steps (native concurrency handles this)
- Reduce six_peer_regression timeout from 120m to 30m
- Skip cross-compile on PRs (only run on main/tags)

Estimated savings: ~50% reduction in CI costs by using standard runners
instead of larger runners, and avoiding duplicate work.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@sanity sanity enabled auto-merge November 30, 2025 16:57
sanity and others added 5 commits November 30, 2025 11:25
Nova has 128GB RAM and fast network, configured with mold linker and
sccache for fast builds. Eliminates GitHub Actions billing entirely.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Jobs now run in order from fastest to slowest:
1. Conventional Commits (~5s)
2. Fmt (~10s)
3. Clippy (~3-5min)
4. Test (~10-15min)
5. six-peer tests (after Test)

This ensures simple issues like formatting or commit messages
fail immediately rather than waiting for long builds.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Add `needs` dependencies to chain jobs in order of execution time:
1. Conventional Commits (fastest, ~5s)
2. Fmt (needs Conventional Commits, ~10s)
3. Clippy (needs Fmt, ~3-5min)
4. Test (needs Clippy, ~10-15min)
5. six-peer tests (needs Test)

With a single self-hosted runner, this ensures faster checks run first
and fail fast, avoiding wasted time on slow tests when simple checks
would fail anyway.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Remove /tmp/freenet and /tmp/freenet-* directories before running tests
to prevent permission denied errors when the self-hosted runner user
cannot access directories created by other users.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@sanity sanity added this pull request to the merge queue Nov 30, 2025
Merged via the queue into main with commit f2a8200 Nov 30, 2025
16 of 17 checks passed
@sanity sanity deleted the ci/reduce-costs branch November 30, 2025 20:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants