Skip to content

ci: bring up what init generates and assert the database persists - #192

Merged
Bccorb merged 1 commit into
mainfrom
ci/scaffold-smoke
Sep 5, 2026
Merged

ci: bring up what init generates and assert the database persists#192
Bccorb merged 1 commit into
mainfrom
ci/scaffold-smoke

Conversation

@Bccorb

@Bccorb Bccorb commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

Closes #166

The gap

Every job in CI asserts the generated files as strings. Nothing executed them, so a compose file could be well-formed and wrong and still pass Build and unit test, pass verify / verify, pass review, and merge. That is how #164 shipped: the harness's own postgres service mounts no volume, so the conformance matrix exercised the in-image default rather than the broken mount, and it was only found when someone ran docker compose up by hand.

What this adds

scripts/scaffold-smoke.sh, run by a new workflow on every PR:

  1. init smoke --local --yes --email=... --auth=docker --admin=none
  2. docker compose up -d --wait db auth
  3. write a row, down (volume kept), up again, read it back
  4. down -v, always, via a trap

Only db and auth come up. Both have healthchecks, so --wait means something, and neither needs an npm install. Bringing up api and web would mostly measure the templates and roughly triple the runtime; that is the "first cut" the issue suggests.

Verified against three real scenarios

The issue asks for confirmation that a deliberate regression fails the job, on the grounds that a smoke test which cannot fail is worse than none. Run locally against Docker:

scenario bring-up result
A. current main healthy PASS, exit 0
B. volume mounted somewhere unused all healthy exit 1 at read-back
C. the #164 mount (/var/lib/postgresql/data) container exits exit 1 at up --wait

B is the one that justifies the design. Bring-up passes completely, four containers report healthy, and the failure only appears at the read-back:

FAIL: the database did not keep its data across a container recreate.
      Expected 42, read ''.
      The generated compose file declares a volume the image does not store data in.

Worth correcting one detail in the issue, though: it predicts C would also start and silently not persist. On postgres:18 the container actually refuses to start, and up --wait catches it. The persistence check is still what earns its place, because it is the only thing that catches B.

Two properties of the generated compose this surfaced

Both are pre-existing and neither is changed here, but they shaped the script:

  • Fixed container_name. seamless-db and seamless-auth are pinned, so -p <project> does not isolate a run. A leftover container produces a raw Docker name conflict, so the script preflights and says what is wrong. It deliberately does not remove containers it did not create; on a developer's machine those are their work.
  • Fixed published ports. The reason the job gets its own runner rather than sharing with verify. SMOKE_EXTRA_COMPOSE lets a developer supply an override to run it beside an existing stack, which is how A, B and C above were exercised on a machine already using 5432.

Flakiness

init fetches the template registry, the templates archive, and the auth .env.example. That failed intermittently during development, always recovering, so the scaffold step retries up to four times with exponential backoff. This job should fail for what the compose file does, not for GitHub being briefly unavailable.

On the seamless check exit code

The issue offers two branches: give check a non-zero exit, or have the smoke job assert independently. This takes the second, so the job does not depend on that decision.

The first is still worth doing, check is what anyone would reach for in a health-check script and it always exits 0, so I am sending it as a separate PR adding --strict rather than mixing a user-facing CLI flag into a CI change. It touches all thirteen check sites and deserves its own review.

Verification

npm run build and npm test pass. AGENTS.md documents the job, why bring-up is not the assertion, and how to run it locally.

Every other job asserts the generated files as strings, so nothing executed
them. A compose file could be well-formed and wrong and still pass CI, review,
and the conformance matrix, which is how the postgres:18 mount regression in
#164 shipped: the harness's own postgres mounts no volume, so it exercised the
in-image default rather than the broken mount.

Adds scripts/scaffold-smoke.sh and a workflow that runs it on every PR. It
scaffolds with --local --yes --auth=docker --admin=none, brings up db and auth
from the generated compose, writes a row, recreates the container with the
volume kept, and reads it back.

Bring-up alone is not the assertion. A volume mounted where the image does not
store data leaves a database that starts, reports healthy, and silently writes
to the container layer, so the read-back is what catches that class.

The scaffold step retries with backoff: init downloads the template registry,
the templates archive and the auth .env.example, and this job is about the
generated compose rather than GitHub's availability.

Closes #166
@Bccorb
Bccorb merged commit 7d0c82f into main Sep 5, 2026
3 checks passed
@Bccorb
Bccorb deleted the ci/scaffold-smoke branch September 5, 2026 18:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test: nothing executes what init generates, so a broken scaffold passes CI

1 participant