Skip to content

ci: add test-nix Linux job using Nix devshell (W50 PR-B)#81

Merged
chaploud merged 1 commit intomainfrom
develop/ci-nix-installer-linux
Apr 29, 2026
Merged

ci: add test-nix Linux job using Nix devshell (W50 PR-B)#81
chaploud merged 1 commit intomainfrom
develop/ci-nix-installer-linux

Conversation

@chaploud
Copy link
Copy Markdown
Contributor

Second step in W50 / Plan B sub-3 (CI Nix-ify). Promotes `scripts/gate-commit.sh` from a developer-only entrypoint to the Linux CI Commit Gate, run inside the Nix devshell.

The flake.nix devshell already provides version-pinned Zig (0.16.0), WASI SDK 30, wasm-tools 1.246.1, wasmtime 42.0.1 (last two pinned via PR-A #80), TinyGo, Go, hyperfine — i.e. the same toolchain Mac/Linux developers get from `direnv allow`. CI now runs that exact toolchain instead of a parallel set of per-tool install steps.

Rust still comes from the runner image's pre-installed rustup; that mirrors the local-developer story (`rustup target add wasm32-wasip1` on top of the Nix devshell). PATH inheritance through `nix develop --command` carries `~/.cargo/bin` into the shell.

What changed

  • New `test-nix` job (ubuntu-latest only): `DeterminateSystems/nix-installer-action` + `magic-nix-cache-action` + `nix develop --command bash -c 'bash scripts/gate-commit.sh'`. Sanity-prints `zig / wasm-tools / wasmtime / WASI_SDK_PATH` versions before the gate so failures are easy to attribute.
  • Existing `test` job matrix narrowed from `[ubuntu, macos, windows]` → `[macos, windows]`. macOS migrates in PR-C, Windows in PR-D (via `pwsh windows/install-tools.ps1`).
  • size-matrix and benchmark jobs untouched (separate jobs; their nix-ification is out of scope here).

Why split macOS off into PR-C

`magic-nix-cache-action` had a 2025 outage and `macos-latest + nix-installer-action` has occasional flakes per the W50 design notes. Linux is the stablest path; landing it first means PR-C/D inherit a known-good `test-nix` shape and any flake-induced failure stays local to its own PR.

Test plan

CI is the test:

  • `flake.nix` already proven across all 4 systems via `nix flake check --all-systems --no-build` (PR-A feat(flake): explicit URL+sha256 pins for wasm-tools + wasmtime (W50 PR-A) #80)
  • New `test-nix` job green on Linux
  • Old `test` job (now macos+windows only) still green — sanity that the matrix shrink didn't break anything
  • No regressions in other jobs (size-matrix, benchmark, versions-lock-sync)

If this PR fails

Stop conditions per session: two non-flaky failures on the same PR → leave as draft, move on. Likely failure modes:

  1. `magic-nix-cache-action` cold start / quota issues — usually self-resolves on rerun.
  2. `nix develop` env not picking up `~/.cargo/bin` — visible in the sanity-version block; would need a small env adjustment.
  3. gate-commit.sh hitting a step that needs a tool the devshell doesn't ship — visible from the failed-step name in the gate summary line.

Second step in W50 / Plan B sub-3 (CI Nix-ify). Promotes
`scripts/gate-commit.sh` from a developer-only entrypoint to the
Linux CI Commit Gate, run inside the Nix devshell.

The flake.nix devshell already provides version-pinned Zig (0.16.0),
WASI SDK 30, wasm-tools 1.246.1, wasmtime 42.0.1 (the last two
pinned via PR-A's URL+sha256 entries), TinyGo, Go, hyperfine — i.e.
the same toolchain Mac/Linux developers get from `direnv allow`. CI
now runs that exact toolchain instead of a parallel set of per-tool
install steps.

Rust still comes from the runner image's pre-installed rustup; that
mirrors the local-developer story (`rustup target add wasm32-wasip1`
on top of the Nix devshell). PATH inheritance through `nix develop
--command` carries `~/.cargo/bin` into the shell.

- New `test-nix` job (ubuntu-latest only): nix-installer-action +
  magic-nix-cache-action + `nix develop --command bash -c
  'bash scripts/gate-commit.sh'`. Sanity-prints `zig / wasm-tools /
  wasmtime / WASI_SDK_PATH` versions before the gate so failures are
  easy to attribute.
- Existing `test` job matrix narrowed from
  `[ubuntu, macos, windows]` → `[macos, windows]`. macOS migrates
  in PR-C, Windows in PR-D (via `pwsh windows/install-tools.ps1`).
- size-matrix and benchmark jobs untouched (separate jobs; their
  nix-ification is out of scope here).

CI-only test plan — no behaviour change for embedders or local devs.
@chaploud chaploud merged commit b23a013 into main Apr 29, 2026
8 checks passed
@chaploud chaploud deleted the develop/ci-nix-installer-linux branch April 29, 2026 07:30
chaploud added a commit that referenced this pull request Apr 29, 2026
Third step in W50 / Plan B sub-3 (CI Nix-ify). Adds macOS to the
Nix-based test job that landed in PR-B (#81). Same flake.nix
devshell, same gate-commit entrypoint, same Rust-from-runner
arrangement — only the runner OS changes.

- test-nix matrix expanded from `[ubuntu-latest]` →
  `[ubuntu-latest, macos-latest]`. Job display name uses
  `${{ matrix.os }}` so the GitHub UI distinguishes the two.
- test job matrix narrowed from `[macos-latest, windows-latest]` →
  `[windows-latest]`. PR-D migrates Windows.
- Zig cache key already used `runner.os`, so `Linux` and `macOS`
  caches stay separate naturally; updated the literal `Linux`
  prefix to `runner.os` to match.

CI-only test plan — no behaviour change for embedders.

Risk note: macos-latest + nix-installer-action has had occasional
flakes in the past per W50 design notes. PR-B's success on Linux
gives confidence the gate-commit path inside the devshell is sound;
if macOS-specific flake behaviour surfaces, leave the PR as draft
and surface for inspection rather than retry-spam.
chaploud added a commit that referenced this pull request Apr 29, 2026
chaploud added a commit that referenced this pull request Apr 29, 2026
Third step in W50 / Plan B sub-3 (CI Nix-ify). Adds macOS to the
Nix-based test job that landed in PR-B (#81). Same flake.nix
devshell, same gate-commit entrypoint, same Rust-from-runner
arrangement — only the runner OS changes.

- test-nix matrix expanded from `[ubuntu-latest]` →
  `[ubuntu-latest, macos-latest]`. Job display name uses
  `${{ matrix.os }}` so the GitHub UI distinguishes the two.
- test job matrix narrowed from `[macos-latest, windows-latest]` →
  `[windows-latest]`. PR-D migrates Windows.
- Zig cache key already used `runner.os`, so `Linux` and `macOS`
  caches stay separate naturally; updated the literal `Linux`
  prefix to `runner.os` to match.

CI-only test plan — no behaviour change for embedders.

Risk note: macos-latest + nix-installer-action has had occasional
flakes in the past per W50 design notes. PR-B's success on Linux
gives confidence the gate-commit path inside the devshell is sound;
if macOS-specific flake behaviour surfaces, leave the PR as draft
and surface for inspection rather than retry-spam.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant