Repository navigation
CI Ubuntu Runners
Status, 2026-11-08. These four runners are no longer registered to this repository. They were re-registered on 2026-11-08, on the same host with the same isolation and limits, to the private mirror described in the owner lane, with the extra label
owner-ci. They now run beside two runners on the owner's Mac (Mac VM runners). Thegh api repos/chris-dare-dev/derived-alg-geo-lean/actions/runnersinspect command below now correctly returns none; inspect the mirror's runners withgh api repos/chris-dare-dev/derived-alg-geo-lean-ci/actions/runnersinstead. "Not instructions to re-register these runners" still holds for this repository.
The owner retired the Windows workstation setup on 2026-09-21 and authorized four runners on the personal Ubuntu PC. Windows is no longer a required platform for this repository. This operator-directed replacement is distinct from the hosted-Linux benchmarking and broader CI1 rollout in #1436. The hosted-only cutover supersedes this scheduling topology after the four repository registrations are removed and the reviewed workflow change merges. The table and operational measurements below document the pre-cutover host; they are not instructions to re-register these runners.
| Runner | Custom scheduling label | Work |
|---|---|---|
| chris-ubuntu-main-runner | owner-linux-main | main push and main manual runs |
| chris-ubuntu-runner-1 | owner-linux | manual runs on other refs |
| chris-ubuntu-runner-2 | owner-linux | manual runs on other refs |
| chris-ubuntu-runner-3 | owner-linux | manual runs on other refs |
All also have the standard self-hosted, Linux, X64 labels and a unique
runner-name label for operator smoke tests. No general runner carries the main
label. PR and merge-group builds continue on ubuntu-latest; branch pushes
alone do not trigger CI. Existing ci protection stays in place. No runner targets the retired employer-issued Mac.
Installation root: ~/.local/share/github-runners/. The four children are
main, general-1, general-2, and general-3. Each owns a distinct home/,
home/.elan/, _work/, tmp/ and cache/. There are no writable cache or
package symlinks between runners or into developer checkouts. Downloads may be
copied, never linked into another runner's mutable build tree.
The host has 16 logical CPUs and approximately 60 GiB RAM. Initial policy:
LEAN_NUM_THREADS=3 per self-hosted build, systemd CPUQuota=300% per service,
MemoryHigh=10G and MemoryMax=12G per service. The aggregate runner slice is
limited to CPUQuota=1200%, MemoryHigh=40G and MemoryMax=48G. These are initial
capacity limits, not benchmarked throughput guarantees. An oversized job may
fail at its memory cap; use actual peak measurements before raising concurrency
or limits. Four registrations are four processes on one physical machine.
At 05:24 UTC, the GitHub runner API reported all four registrations online;
the main runner was busy and the three general runners idle. systemctl --user
reported four active services with separate main PIDs. Each runner's
_work/derived-alg-geo-lean/derived-alg-geo-lean, home/.elan, tmp, and
cache resolved under its own
/home/chris-dare/.local/share/github-runners/{main,general-1,general-2,general-3}/
root. The four resolved .lake/packages paths were distinct directories, not
symlinks. This is a point-in-time inventory, not a proof about future jobs.
The host measured 16 logical CPUs, 65,374,400,512 bytes of RAM and 141,614,346,240 bytes of available filesystem space. The live service settings were three CPU seconds per second and 12 GiB maximum memory each; the shared runner slice was capped at 12 CPU seconds per second and 48 GiB. Those values bound the starting host-wide reservation policy, subject to fresh free-space/load readings and throughput measurements. Do not multiply capacity by the four runner labels.
A separate read-only scan of this clone's 219 registered agent worktrees found
three resolved .lake/packages targets shared by multiple worktrees (groups
of 38, 22 and 19). These are not the runner package directories. They must
not be treated as immutable merely because a worktree's .lake/packages is a
symlink; the new admission preflight rejects such paths for a writable job.
Migrate an idle agent non-destructively: create a new isolated worktree and
job-owned package tree, copy from an idle verified donor if needed, compare
the pin/toolchain inputs, run the focused build, then switch the pickup path.
Keep the old checkout and cache until the new one is verified. No active
service, checkout or cache was changed for this inventory.
A read-only rescan on 2026-09-28 found 246 registered worktrees, 223 existing
paths, and 107 .lake/packages links resolving to four shared targets. These
are existing user worktrees; neither the runner installation nor the new
worktree seeder deletes or rewrites them. On an idle migration, preserve the
old worktree, create a new worktree at its commit, and invoke the private-cache
helper from an updated checkout containing this change, with the new worktree
as its explicit --target:
SAFE_ROOT=/absolute/path/to/updated/derived-alg-geo-lean
NEW=/absolute/path/to/new-worktree
python3 "$SAFE_ROOT/scripts/private_package_cache.py" --target "$NEW" --dry-run
python3 "$SAFE_ROOT/scripts/private_package_cache.py" --target "$NEW"Do not run the seeder script inside a worktree at an older commit: that
commit may still contain the link-producing implementation. The current helper
requires private package/build caches and verifies their receipt. Compare the
new source and Git index with the old worktree, carry any uncommitted and
untracked work deliberately, and move the pickup path only after verification.
An old linked .lake refuses seeding in place, including with --force.
Each migrated worktree needs its own disk space; the current seeder checks
headroom and refuses before publication when it is insufficient.
The updated helper reads pinned package source through the tracked minimal
scripts/private-reader.git and rechecks source bytes independently of Git's
index flags. It scans the whole private .lake for hardlinks, mount aliases
and symlinks other than the exact tracked package-source links before
publication and on reuse. A build may change its
private Git indexes and build output, but a tracked source edit refuses reuse.
The supported post-build check is the targeted Mathlib build; tracked
ProofWidgets/npm outputs retain their pinned bytes. This is a quiescent pickup
check among cooperating processes, not a barrier against later same-account
mutation. It refuses ambiguous bind-mounted worktree ancestry rather than
repointing or repairing it.
The user services invoke the runner's runsvc.sh entrypoint and restart after
failures. User lingering makes them start at boot and survive logout. Inspect:
systemctl --user status dag-runner@main dag-runner@general-{1,2,3}
journalctl --user -u dag-runner@main -n 80
loginctl show-user "$USER" -p Linger
gh api repos/chris-dare-dev/derived-alg-geo-lean/actions/runnersServices intentionally use runner-owned HOME values and a minimal PATH; the owner's gh keyring/config and personal Lean installation are not copied. Jobs still run as the same Unix account: separate directories and cgroup budgets are operational isolation, not a security boundary against hostile code.
Wait for an idle runner before stopping or changing its service. On a failed
job, inspect its log and service memory events; do not erase a sibling's state.
Stop/start a specific service with systemctl --user stop/start dag-runner@NAME.
Systemd's control-group shutdown reaps that service's child processes. Runner
auto-updates retain the same directory and service identity.
Required host packages include build-essential, python3-venv, python3-pip,
pkg-config, libgmp-dev, unzip, zstd and the runner's ICU/OpenSSL runtime
libraries. Lean setup follows the committed toolchain through lean-action.
gh must come from GitHub's apt repository, not Ubuntu's archive, whose
2.46 fails roadmap (see the owner lane). Since
2026-11-08 this host has gh 2.103.3 from https://cli.github.com/packages.
If routing must be rolled back, use hosted Ubuntu through a reviewed change; do not recreate the repository's Ubuntu or former Windows registrations. This migration does not claim completion of CI1 host-admission tooling, trusted-cache consolidation, cold/warm parity or the week-long rollout study.
Exported from docs/ci/ubuntu-runners.md at the commit that removed it from the tree.
Architecture
- Canonical roots and specialization trees
- Structural cutover ledger
- Generalization backlog
- Dependency direction
- Loop-engineering friction log
- Mathematical ownership and abstraction boundarie
- Declaration placement
- When a proposition-valued class is justified
- Enhanced spherical twists: conventions and remai
- Verification history
CI and runners
- Bounded audit batching probe, 2026-09-27
- CI1.10: audit and exception record contention, 2
- Proposed hosted-only CI cutover for CI1.07
- Mac VM runners
- Owner CI lane: self-hosted runners in a private
- CI performance baseline, 2026-09-27
- CI1.07 exact-tree Ubuntu routing sample, 2026-09
- Ubuntu workstation runners (historical topology)
Decision records
- ADR-0001 — The seam is cold: files, not runtime (superseded)
- ADR-0002 — Identity is minted in the topic repo, (superseded)
- ADR-0003 — Elaboration and axiom evidence is pro (superseded)
- ADR-0004 — The statement registry lives in the t (superseded)
- ADR-0005 — Seven axes, four values, no aggregate (superseded)
- ADR-0006 — All contract issues are filed on `bri (superseded)
- ADR-0007 — Where the contract package lives (superseded)
-
ADR-0008 —
@[cites]ships as a shared Lake dep (superseded) - ADR-0009 — The contract package lives with the e (superseded)
- ADR-0010 — How a dg category is encoded, and whe
- ADR-0011 — Hom-complexes are complexes of abelia
- ADR-0012 — One Mathlib-style source root
- ADR-0013 — Own reusable moduli infrastructure in
- ADR-0014 — Stacks, algebraic spaces and moduli:
Roadmaps