ci(e2e): wire the dogfood full-stack e2e suite as a per-PR gate by seal-agent · Pull Request #256 · RigelBuild/compass

seal-agent · 2026-08-10T01:38:23Z

This PR is part of a stack containing 2 PRs:

main
test(e2e): gate the dogfood fixture on runner enrollment #270
"ci(e2e): wire the dogfood full-stack e2e suite as a per-PR gate" (this PR)

Wires the //go:build podman dogfood e2e suite (go/e2e/...) as a required per-PR check in GitHub Actions, so the full-stack deterministic tier — real compass-server + runner + agent container + a private postgres, driven by a canned in-process model — runs on every PR rather than only inside a dev shell. This is task H8 of the dogfood-e2e design record, and it lands the second of SEA-1359's two portability proofs (the first, the focused Launch-path invariant, merged as #247): a green run here proves the stack stands up on an ordinary Nix-free ubuntu-latest runner.

The postgres-on-PATH fix

The e2e harness's private postgres (go/compass-postgres/main.go) shells out to initdb / postgres / createdb via exec.LookPath. In the dev shell those come free from services.postgres, but CI's gate-tools (tools/toolchain/gate-tools.nix, fed by parity.ts --print-nix-attrs) builds its PATH env only from the devenv packages list — a devenv service never reaches a CI runner. So a naive wiring reds at stack.Up with locating initdb binary: executable file not found in $PATH.

Fix: add bare postgresql to devenv.nix packages. Bare (not a version-suffixed attr) for strict parity — services.postgres.package defaults to bare pkgs.postgresql, which at this devenv.lock pin resolves to postgresql-18.4, so CI and the dev shell exercise one postgres derivation. The toolchain-parity gate confirms all of postgres's binaries (including initdb) resolve to that pinned store path.

Image source — test the image the PR would produce

The e2e fixture resolves compass-agent:latest from local containers-storage (no pull at test time). The seed step decides where that tag comes from per-PR:

image-input-changing PR → build+load from this tree: nix run path:../forks/devenv#devenv -- container copy agent (from agent-image/), so the gate proves the image this PR produces, not the last-published :latest. No registry round-trip, no :pr-<N> tag — the fixture reads local storage.
otherwise (and every push to main) → pull the published ghcr.io/rigelbuild/compass-agent:latest.

"Image inputs changed" is detected via moon query projects --affected --id compass-agent-image, reusing the compass-agent-image moon project's inputs as the single source of truth for the image closure — so the detection can't drift from what actually rebuilds the image. :latest is kept mutable deliberately (always-fresh), not digest-pinned.

The assert-ran guard

The e2e legs t.Skip when rootless podman is unavailable. A required check that let that skip pass would be vacuously green, so a guard step derives the skip string from go/e2e/harness_test.go and the ok line for the e2e package specifically, and reds on a silent skip. The skip-string half is load-bearing: go/e2e/cannedmodel_test.go is deliberately untagged and runs under a bare go test, so an ok-line check alone could pass on that file while the real podman legs skipped.

Design record

Amends §H8 of docs/designs/platform/compass-dogfood-e2e/design.md to the landed end state, and syncs three now-stale references across the record (Global Constraints, Decision D2, the H8 task line): SEA-1690 (public GHCR image) and SEA-1691 (host-uid lift, the userns keep-id remap that replaced verifyRunnerUID) have both landed, so the full-stack tier runs as the required per-PR check directly on ordinary arbitrary-uid runners — no interim merge-queue/nightly staging. Adds the postgres PATH prereq to the documented runner requirements.

This re-authors an earlier draft of the CI wiring under compass ownership after the compass CI surface moved into the compass agents' lane.

Refs SEA-1792
Refs SEA-1359
Co-authored-by: Matt Wilkinson matt@sealedsecurity.com

linear-code · 2026-08-10T01:38:26Z

SEA-1792

SEA-1359

Review of #256 surfaced a vacuous detection at the head of the e2e gate: `moon query projects --affected --id compass-agent-image` prints a JSON envelope on stdout unconditionally — an unaffected project yields `{"projects": [], "options": {…}}`, not empty output — so `[ -n "$affected" ]` was always true. The seed step therefore always took the build-from-tree branch and the published-`:latest` pull path was dead code on every PR, defeating the point of the detection (and doing a full image build even on PRs that don't touch the image closure). Discriminate on the `.projects` array instead: `jq -e '.projects | length > 0'` is true only when the project is actually affected. `jq` is preinstalled on `ubuntu-latest`; its false-exit is safe inside the `if` condition under the runner's default `bash -e`. Verified in a throwaway git repo against moon 2.4.x: an unaffected `--id` returns `{"projects": []}` (length 0), an affected one returns a populated array. Also from the review: - Scope the seed step's "near-free" cost claim to the image-affected case — now the only case the build branch fires, where `moon ci :ci` already warmed the nix store on this PR. - Correct the private-postgres harness path cite `go/compass-postgres/main.go` → `go/cmd/compass-postgres/main.go` in `devenv.nix` and the design record. Spec-impact: none. Refs SEA-1792. Co-authored-by: Matt Wilkinson <matt@sealedsecurity.com>

…urn delivery Freezes the design for removing `initial_prompt` from the whole Compass start contract (proto -> server -> runner -> agent SDK -> UI -> e2e harness). Matt has ruled the field out of existence: an agent session **always starts idle**, and its first turn arrives through a channel — never a prompt field the Runner silently drops today (`(*agentHost).Start` reads only the container name; every prompt riding the field is dropped and the agent idles forever). This record is the contract that unblocks SEA-1792 H8 / PR #256 — the dogfood e2e `TestLegTwoRealTurn` is red precisely because its first turn rides the dropped field. ### What it decides (DL-186..189, appended to the product ledger by T7) - **DL-186** — `initial_prompt` removed from the whole contract (three fields, numbers + names reserved), consumers cut over atomically with no fallback; first turn arrives over the SEA-1569 deliver path. - **DL-187** — `@compass` reserved alias frozen as the system-sender mechanism for **any** system-level message; case-1 root-manager boot scoped out to follow-up **SEA-1820**. - **DL-188** — fresh-start barrier-lift: on a fresh start the Runner sends `AgentControl{replay_complete}` as the first control op after Bind (symmetric with the resume path, no agent change). - **DL-189** — the case-2 brief carrier is a **per-pair manager↔peer DM channel** (`ChannelKindDM`), auto-provisioned on the spawn edge — not the coordination broadcast, so siblings never receive briefs they don't need. ### Staging - **PR-A** (additive, lands first) — Runner deliver-lane build (`DeliverControl` dispatch arm, `representable()` admits Deliver/Steer, fresh-start barrier-lift) + spawn-edge DM auto-provision. Greens nothing yet; independently revertable. - **PR-B** (lands after PR-A) — the atomic `initial_prompt` removal + e2e harness re-model; greens leg-2. ### Honest state The server + agent halves of the deliver seam exist; the **Runner middle leg is UNBUILT** at three source-verified points (dispatch has no `DeliverControl` arm; gateway `representable()` rejects Deliver/Steer; no fresh-start `replay_complete` sender + the agent replay barrier strands a pre-`replayComplete` deliver). The record does not claim case-2 is verified end-to-end today — PR-A builds the leg. ### Open for Matt at this gate - **[FORK]** `StartAgentDialog` disposition — removing the prompt empties the dialog, but DL-185 (SEA-1932, Active) explicitly *keeps* it. Delete vs bare-confirm is Matt's call (recommendation: delete). One conditional status-flip on DL-185's "Kept" clause. - **[Minor]** settle-primitive naming (`AwaitTurnSettled`). Every cited file:line grounded against `main@origin`. No code, codegen, or proto changed — design record only. Spec-impact: none. Refs SEA-1681, SEA-1820. Co-authored-by: Matt Wilkinson <matt@sealedsecurity.com>

Review of #256 surfaced a vacuous detection at the head of the e2e gate: `moon query projects --affected --id compass-agent-image` prints a JSON envelope on stdout unconditionally — an unaffected project yields `{"projects": [], "options": {…}}`, not empty output — so `[ -n "$affected" ]` was always true. The seed step therefore always took the build-from-tree branch and the published-`:latest` pull path was dead code on every PR, defeating the point of the detection (and doing a full image build even on PRs that don't touch the image closure). Discriminate on the `.projects` array instead: `jq -e '.projects | length > 0'` is true only when the project is actually affected. `jq` is preinstalled on `ubuntu-latest`; its false-exit is safe inside the `if` condition under the runner's default `bash -e`. Verified in a throwaway git repo against moon 2.4.x: an unaffected `--id` returns `{"projects": []}` (length 0), an affected one returns a populated array. Also from the review: - Scope the seed step's "near-free" cost claim to the image-affected case — now the only case the build branch fires, where `moon ci :ci` already warmed the nix store on this PR. - Correct the private-postgres harness path cite `go/compass-postgres/main.go` → `go/cmd/compass-postgres/main.go` in `devenv.nix` and the design record. Spec-impact: none. Refs SEA-1792. Co-authored-by: Matt Wilkinson <matt@sealedsecurity.com>

Wires the `//go:build podman` dogfood e2e suite (`go/e2e/...`) as a required per-PR check in GitHub Actions, so the full-stack deterministic tier — real compass-server + runner + agent container + a private postgres, driven by a canned in-process model — runs on every PR rather than only inside a dev shell. This is task H8 of the dogfood-e2e design record, and it lands the second of SEA-1359's two portability proofs (the first, the focused Launch-path invariant, merged as #247): a green run here proves the stack stands up on an ordinary Nix-free ubuntu-latest runner. ### The postgres-on-PATH fix The e2e harness's private postgres (`go/compass-postgres/main.go`) shells out to `initdb` / `postgres` / `createdb` via `exec.LookPath`. In the dev shell those come free from `services.postgres`, but CI's gate-tools (`tools/toolchain/gate-tools.nix`, fed by `parity.ts --print-nix-attrs`) builds its PATH env only from the devenv `packages` list — a devenv *service* never reaches a CI runner. So a naive wiring reds at `stack.Up` with `locating initdb binary: executable file not found in $PATH`. Fix: add bare `postgresql` to `devenv.nix` `packages`. Bare (not a version-suffixed attr) for strict parity — `services.postgres.package` defaults to bare `pkgs.postgresql`, which at this `devenv.lock` pin resolves to postgresql-18.4, so CI and the dev shell exercise one postgres derivation. The toolchain-parity gate confirms all of postgres's binaries (including `initdb`) resolve to that pinned store path. ### Image source — test the image the PR would produce The e2e fixture resolves `compass-agent:latest` from local containers-storage (no pull at test time). The seed step decides where that tag comes from per-PR: - **image-input-changing PR** → build+load from this tree: `nix run path:../forks/devenv#devenv -- container copy agent` (from `agent-image/`), so the gate proves the image *this PR* produces, not the last-published `:latest`. No registry round-trip, no `:pr-<N>` tag — the fixture reads local storage. - **otherwise** (and every push to main) → pull the published `ghcr.io/rigelbuild/compass-agent:latest`. "Image inputs changed" is detected via `moon query projects --affected --id compass-agent-image`, reusing the `compass-agent-image` moon project's `inputs` as the single source of truth for the image closure — so the detection can't drift from what actually rebuilds the image. `:latest` is kept mutable deliberately (always-fresh), not digest-pinned. ### The assert-ran guard The e2e legs `t.Skip` when rootless podman is unavailable. A required check that let that skip pass would be vacuously green, so a guard step derives the skip string from `go/e2e/harness_test.go` and the `ok` line for the e2e package specifically, and reds on a silent skip. The skip-string half is load-bearing: `go/e2e/cannedmodel_test.go` is deliberately untagged and runs under a bare `go test`, so an `ok`-line check alone could pass on that file while the real podman legs skipped. ### Design record Amends §H8 of `docs/designs/platform/compass-dogfood-e2e/design.md` to the landed end state, and syncs three now-stale references across the record (Global Constraints, Decision D2, the H8 task line): SEA-1690 (public GHCR image) and SEA-1691 (host-uid lift, the userns keep-id remap that replaced `verifyRunnerUID`) have both landed, so the full-stack tier runs as the required per-PR check directly on ordinary arbitrary-uid runners — no interim merge-queue/nightly staging. Adds the postgres PATH prereq to the documented runner requirements. This re-authors an earlier draft of the CI wiring under compass ownership after the compass CI surface moved into the compass agents' lane. Refs SEA-1792 Refs SEA-1359 Co-authored-by: Matt Wilkinson <matt@sealedsecurity.com>

Review of #256 surfaced a vacuous detection at the head of the e2e gate: `moon query projects --affected --id compass-agent-image` prints a JSON envelope on stdout unconditionally — an unaffected project yields `{"projects": [], "options": {…}}`, not empty output — so `[ -n "$affected" ]` was always true. The seed step therefore always took the build-from-tree branch and the published-`:latest` pull path was dead code on every PR, defeating the point of the detection (and doing a full image build even on PRs that don't touch the image closure). Discriminate on the `.projects` array instead: `jq -e '.projects | length > 0'` is true only when the project is actually affected. `jq` is preinstalled on `ubuntu-latest`; its false-exit is safe inside the `if` condition under the runner's default `bash -e`. Verified in a throwaway git repo against moon 2.4.x: an unaffected `--id` returns `{"projects": []}` (length 0), an affected one returns a populated array. Also from the review: - Scope the seed step's "near-free" cost claim to the image-affected case — now the only case the build branch fires, where `moon ci :ci` already warmed the nix store on this PR. - Correct the private-postgres harness path cite `go/compass-postgres/main.go` → `go/cmd/compass-postgres/main.go` in `devenv.nix` and the design record. Spec-impact: none. Refs SEA-1792. Co-authored-by: Matt Wilkinson <matt@sealedsecurity.com>

…urn delivery (#273) * docs(product): design record — remove initial_prompt, channel-first-turn delivery Freezes the design for removing `initial_prompt` from the whole Compass start contract (proto -> server -> runner -> agent SDK -> UI -> e2e harness). Matt has ruled the field out of existence: an agent session **always starts idle**, and its first turn arrives through a channel — never a prompt field the Runner silently drops today (`(*agentHost).Start` reads only the container name; every prompt riding the field is dropped and the agent idles forever). This record is the contract that unblocks SEA-1792 H8 / PR #256 — the dogfood e2e `TestLegTwoRealTurn` is red precisely because its first turn rides the dropped field. ### What it decides (DL-186..189, appended to the product ledger by T7) - **DL-186** — `initial_prompt` removed from the whole contract (three fields, numbers + names reserved), consumers cut over atomically with no fallback; first turn arrives over the SEA-1569 deliver path. - **DL-187** — `@compass` reserved alias frozen as the system-sender mechanism for **any** system-level message; case-1 root-manager boot scoped out to follow-up **SEA-1820**. - **DL-188** — fresh-start barrier-lift: on a fresh start the Runner sends `AgentControl{replay_complete}` as the first control op after Bind (symmetric with the resume path, no agent change). - **DL-189** — the case-2 brief carrier is a **per-pair manager↔peer DM channel** (`ChannelKindDM`), auto-provisioned on the spawn edge — not the coordination broadcast, so siblings never receive briefs they don't need. ### Staging - **PR-A** (additive, lands first) — Runner deliver-lane build (`DeliverControl` dispatch arm, `representable()` admits Deliver/Steer, fresh-start barrier-lift) + spawn-edge DM auto-provision. Greens nothing yet; independently revertable. - **PR-B** (lands after PR-A) — the atomic `initial_prompt` removal + e2e harness re-model; greens leg-2. ### Honest state The server + agent halves of the deliver seam exist; the **Runner middle leg is UNBUILT** at three source-verified points (dispatch has no `DeliverControl` arm; gateway `representable()` rejects Deliver/Steer; no fresh-start `replay_complete` sender + the agent replay barrier strands a pre-`replayComplete` deliver). The record does not claim case-2 is verified end-to-end today — PR-A builds the leg. ### Open for Matt at this gate - **[FORK]** `StartAgentDialog` disposition — removing the prompt empties the dialog, but DL-185 (SEA-1932, Active) explicitly *keeps* it. Delete vs bare-confirm is Matt's call (recommendation: delete). One conditional status-flip on DL-185's "Kept" clause. - **[Minor]** settle-primitive naming (`AwaitTurnSettled`). Every cited file:line grounded against `main@origin`. No code, codegen, or proto changed — design record only. Spec-impact: none. Refs SEA-1681, SEA-1820. Co-authored-by: Matt Wilkinson <matt@sealedsecurity.com> * docs(product): re-pin first-turn record citations to main@origin Review round 1 (sole review-of-record) found 0 high / 2 medium / 3 low, all citation line-number accuracy — no design-judgment findings. The record's file:line were pinned to a superseded base (f8ab68c); the record's actual parent and the ws-first-turn worktree it names as its citation source is main@origin, where SEA-1575 (2ebdf09) restructured dispatch.go and host.go. Re-pinned every drifted citation, ground-verified against main@origin: - dispatch.go execute switch :243 -> :359; default error :320-323 -> :446-449 - host.go Start/container read :264-265 and the wrong :227-228/:227 -> :284-285 - host.go READY status-answer :346,480 -> :405,785 - consumer.go deliverOp :277-281 -> :288-291 (plain miscite, not drift) - runner.pb.go DeliverControl :599,634 -> :618,655 (was pointing at the wrong variant) - mapping.ts agent_end READY :145-146 -> :115-116 Semantics were correct throughout; this is a locational fix only. The Per-site table header already labels citations as ws-first-turn (= main@origin), so the numbers now match the stated base. Additive commit atop the pushed head so the PR shows the interdiff. Refs SEA-1681, SEA-1820 Co-authored-by: Matt Wilkinson <matt@sealedsecurity.com> * docs(product): fold Matt's OQ-1 ruling — delete StartAgentDialog At the design-PR gate Matt ruled OQ-1 (the one genuine fork left in the record): StartAgentDialog is DELETED in T5, not reduced to a bare confirm dialog. Removing initial_prompt empties the dialog (its only input was the prompt textarea); the board start affordance invokes the spawn action directly with {agentAccountId, workstreamId} (spawn is already guarded by the DL-164/168 live-session predicate and idempotent under its request id). Ground check at ruling time: StartAgentDialog.tsx + .test.tsx still present on main@origin and no open PR removes them, so the deletion is this design's own, landed by T5 in PR-B after #267/SEA-1932 merges (so DL-185's "Kept" clause exists to be amended). This resolves the record's last open fork; the status-flip is now "one (Matt-ruled)". Settle-primitive naming (AwaitTurnSettled) was already driver-resolved in-record. Refs SEA-1681, SEA-1820 Co-authored-by: Matt Wilkinson <matt@sealedsecurity.com> --------- Co-authored-by: Matt Wilkinson <matt@sealedsecurity.com>

seal-agent requested a review from mattwilkinsonn as a code owner August 10, 2026 01:38

seal-agent mentioned this pull request Aug 10, 2026

ci(compass): gate PRs on the dogfood e2e podman tier (SEA-1792) #246

Closed

seal-agent force-pushed the seal-1792-h8-e2e-ci-gate branch from ef83448 to 1816be7 Compare August 10, 2026 02:17

seal-agent mentioned this pull request Aug 10, 2026

docs(product): design record — remove initial_prompt, channel-first-turn delivery #273

Merged

seal-agent changed the base branch from main to seal-h8-enroll-gate August 10, 2026 20:12

seal-agent force-pushed the seal-1792-h8-e2e-ci-gate branch from 1816be7 to 15e0d26 Compare August 10, 2026 20:12

seal-agent mentioned this pull request Aug 10, 2026

test(e2e): gate the dogfood fixture on runner enrollment #270

Merged

Base automatically changed from seal-h8-enroll-gate to main August 12, 2026 19:16

seal-agent and others added 2 commits August 12, 2026 15:16

mattwilkinsonn force-pushed the seal-1792-h8-e2e-ci-gate branch from 15e0d26 to 91fd32b Compare August 12, 2026 19:16

Navigation Menu

Uh oh!

ci(e2e): wire the dogfood full-stack e2e suite as a per-PR gate - #256

ci(e2e): wire the dogfood full-stack e2e suite as a per-PR gate#256
seal-agent wants to merge 2 commits into
mainfrom
seal-1792-h8-e2e-ci-gate

seal-agent commented Aug 10, 2026 •

edited

Loading

Uh oh!

linear-code Bot commented Aug 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

seal-agent commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

The postgres-on-PATH fix

Image source — test the image the PR would produce

The assert-ran guard

Design record

Uh oh!

linear-code Bot commented Aug 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

seal-agent commented Aug 10, 2026 •

edited

Loading