ci: pre-deploy smoke gate (Option C) so a JS-dead render can't reach isamples.org#225
Merged
Merged
Conversation
The deploy workflow runs `quarto render` and ships whatever docs/ it produces; nothing ever loads the rendered page in a browser, so a render that "succeeds" but yields a page where DuckDB-WASM never inits, Cesium never draws, or search returns nothing has historically deployed anyway (the failure class behind past "reviewed and still broke" incidents). Adds tests/test_smoke.py: single fresh context, one navigation, poll-for-readiness (no reload loop — rapid reloads exhaust the DuckDB-WASM worker and false-fail). Asserts four unambiguous liveness signals: facet query populated, Cesium canvas attached, a world search returns results, no uncaught JS exception / regression-fingerprint console error. Wires it into quarto-pages.yml between render and Deploy, serving the rendered docs/ locally. Fail-closed: smoke failure fails the job and the Deploy step is skipped. trap-reaps the static server under `bash -e`. Validated both directions: passes the known-good build in ~15s; raises TimeoutError on a rendered-but-JS-dead page. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…g#225 - Scope _FATAL_CONSOLE to same-origin scripts: a third-party console.error (Cesium CDN, injected extension) can no longer block a deploy; pageerror stays the unconditional hard signal for uncaught app exceptions. - Cesium check now also asserts non-zero canvas dimensions, catching the "widget mounted but globe never sized" case without flaky pixel readback. - Search-result wait 60s -> 90s, aligned with the perf test budget, so a slow CI cold DuckDB-WASM query + remote parquet fetch doesn't false-fail a healthy build. Re-validated: passes the known-good build (~20s). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
Author
How to test this gate / what to look forRun it exactly as CI willgit fetch origin smoke-gate-option-c && git checkout smoke-gate-option-c
python3 -m http.server 8080 --directory docs &
ISAMPLES_BASE_URL=http://localhost:8080 pytest tests/test_smoke.py -s -q
kill %1Expected (healthy build): Prove it's fail-closed (catches a break)cp docs/explorer.html /tmp/explorer.bak
sed -i '' "s/data-value='SESAR'/data-value='BROKEN'/g" docs/explorer.html
ISAMPLES_BASE_URL=http://localhost:8080 pytest tests/test_smoke.py -s -q # -> 1 failed (TimeoutError on readiness)
cp /tmp/explorer.bak docs/explorer.htmlWhat each assertion proves
Review sanity-checks
CI-only difference from the local run: CI first does |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The deploy workflow runs
quarto renderand ships whateverdocs/it produces. Neither code review norpytest --collect-onlyever loads the rendered page in a browser, so a render that "succeeds" but yields a JS-dead explorer (DuckDB-WASM never inits, Cesium blank, search returns nothing) deploys to isamples.org anyway. This is the failure class behind past "we reviewed it and it still broke" incidents.What this adds
tests/test_smoke.py— fundamental-liveness gate. Single fresh context, one navigation, poll-for-readiness (deliberately no reload loop; rapid reloads exhaust the DuckDB-WASM worker and produce false failures). Four unambiguous assertions:#searchSubmitBtnreturns resultsquarto-pages.yml— smoke step inserted betweenquarto renderandDeploy, serving the rendereddocs/locally. Fail-closed: smoke failure fails the job, soDeployis skipped and a broken render never reaches production.trap-reaps the static server under GitHub'sbash -e.Validation
TimeoutErroron a rendered-but-JS-dead page → pytest fails → step fails → Deploy skipped (fail-closed confirmed).Notes
main(the deploy trigger), so it self-validates on its first real deploy after merge. Worst case of a false-fail is a blocked deploy, not a broken site.🤖 Generated with Claude Code