Fix broken GitHub Pages deploy: substitute {BASE_PATH}, CORS-aware disk probe, debug mode, Pages-parity tests (#3) by konard · Pull Request #4 · link-foundation/rust-web-box

konard · 2026-04-29T16:44:53Z

Fixes #3.

Problem

The site at https://link-foundation.github.io/rust-web-box/ rendered an empty workbench: no terminal, no loading indication, no VM health, no files. The deeply unsatisfying part was that everything worked on a local python -m http.server — so the failure surfaced only after a deploy.

Root causes (3, ordered by severity)

#	Cause	Severity	Fix
1	`additionalBuiltinExtensions[].path` was a host-absolute URL path, but the page is served from `/rust-web-box/`. The browser asked for `https://host/extensions/...` and 404'd silently. With no FileSystemProvider for `webvm:`, no terminal profile, and no extension code running, the workbench rendered an empty Welcome page.	P0	`9ada2ea` — substitute `{BASE_PATH}` placeholder in both the inline bootstrap (HTML) and the defensive backstop in `glue/boot.js`.
2	The warm disk-image probe used `mode: 'no-cors'` and treated opaque responses as "OK" — telling us "the URL is fetchable" but NOT "the URL is JS-readable". We then handed CORS-blocked URLs to `CloudDevice.create()` and only failed at mount time.	P1	`2cfcbe4` — probe with `mode: 'cors'`, return structured `{ok, reason}`, and surface a diagnostic pointing to the case-study doc.
3	When a maintainer hits a regression like this, the only signal is a blank screen and three obscure console errors. No way to ask "what state is the boot pipeline in right now?"	P2	`e27fe30` — opt-in verbose mode (`?debug=1` / `localStorage.rustWebBoxDebug=1`), with namespace filtering and a `__rustWebBox.dump()` snapshot for bug reports. Zero overhead when off.

A fourth contributing cause — GitHub Pages doesn't emit COOP/COEP — is already worked around by the existing service worker; the pages-parity test pins the headers so a refactor can't drop them.

Acceptance criteria from the issue

Visual proof

Before (empty workbench, only console errors):

After (workbench mounted with hello_world.rs open, terminal active, files visible):

Captured by Playwright against the dev server in sub-path mode (--base=/rust-web-box), which mirrors the GitHub Pages deployment topology exactly.

Test plan

$ node --test web/tests/
# tests 119
# pass 119
# fail 0

New test files (24 new tests):

web/tests/disk-cors.test.mjs — 11 tests pinning CORS probe semantics, structured diagnostics, legacy boolean tolerance.
web/tests/debug-mode.test.mjs — 11 tests for the verbose-mode parser, namespace filtering, dumpRuntime snapshot.
web/tests/pages-parity.test.mjs — 5 tests spawning the dev server in sub-path mode and verifying COOP/COEP headers, extension reachability, {BASE_PATH} substitution, and outside-prefix 404s.
web/tests/cargo-install.test.mjs — 4 tests exercising the full cargo install serde request sequence through the production network-shim routing.

Tooling:

web/build/dev-server.mjs gained --base=/path, mirroring the Pages topology so this class of bug can be reproduced locally before deploy.

Reproduction

To reproduce the original bug from a clean checkout:

node web/build/build-workbench.mjs
node web/build/dev-server.mjs 8080 --base=/rust-web-box
# open http://localhost:8080/rust-web-box/

Without the 9ada2ea fix, the workbench renders empty and the console shows 404s for /extensions/webvm-host/package.json. With the fix, it loads as in the "after" screenshot.

To turn on verbose mode for any future regression:

http://localhost:8080/rust-web-box/?debug=1
http://localhost:8080/rust-web-box/?debug=boot,workbench,cheerpx

Then in DevTools: __rustWebBox.dump() returns a JSON-safe runtime snapshot.

Case study

Full timeline, root-cause analysis, evidence, and online research collected in docs/case-studies/issue-3/:

README.md — narrative + recommendations
analysis-extension-base-path.md — root cause MVP: anonymous in-browser Rust sandbox (full VS Code Web + WebVM + crates.io) on GitHub Pages #1
analysis-disk-cors.md — root cause MVP foundation: static shell + cargo network shim + Pages deploy (#1) #2
analysis-coop-coep.md — root cause Fix all bugs, and improve quality of our project #3 (already mitigated by SW; pinned by tests)
online-research.md — prior art from leaningtech/webvm and microsoft/vscode

Commits

e89814f docs(case-study): deep dive into issue #3 (broken Pages deploy)
9ada2ea fix(web): substitute {BASE_PATH} so extensions resolve under sub-paths (#3)
2cfcbe4 fix(web): probe warm disk under cors mode so we detect XHR-blocked URLs (#3)
e27fe30 feat(web): opt-in verbose debug mode (?debug=1) and __rustWebBox.dump() (#3)
6040569 test(web): Pages-parity dev server + sub-path deployment test (#3)
4e3f699 test(web): cargo-install end-to-end via network shim (R7, #3)

Adding .gitkeep for PR creation (default mode). This file will be removed when the task is complete. Issue: #3

Compile a full case study for issue #3 documenting why https://link-foundation.github.io/rust-web-box loaded an empty workbench: timeline, requirements R1-R11, three ranked root causes (extension URL base path, warm-disk CORS, top-level COOP/COEP), proposed fixes per cause, upstream reports to file, and an external research log of similar projects (JupyterLite, Wasmer, coi-serviceworker). Evidence captured under docs/case-studies/issue-3/evidence/ (devtools console logs + curl headers) so future maintainers can re-verify the analysis without re-running the deploy.

#3) additionalBuiltinExtensions[].path is a host-absolute URL path, so on GitHub Pages (where the document is served at /rust-web-box/) the workbench was requesting /extensions/webvm-host/package.json — missing the deploy-base prefix — and silently 404ing. With no FileSystemProvider for webvm: registered, the workbench fell back to an empty Welcome page: no terminal, no files, no VM status. This is root cause #1 of issue #3. Substitute a {BASE_PATH} placeholder (derived from location.pathname) in both the inline bootstrap (which runs first) and boot.js (defensive backstop). A belt-and-braces branch also rewrites legacy paths without the placeholder so a stale build artifact cannot silently re-introduce the bug. Extracted the substitution logic into glue/workbench-config.js so Node tests can import it without pulling in browser globals (location/document). Tests: 9 new cases in tests/extension-paths.test.mjs covering dev-server root, GitHub Pages, deeply nested sub-paths, idempotency, and the no-placeholder legacy path. All passing.

…Ls (#3) The previous probe used mode:'no-cors' and treated an opaque response as success. That told us "the URL is fetchable" but NOT "the URL is JS-readable", so we happily handed CORS-blocked URLs to CloudDevice.create() — which uses XHR (no no-cors mode) and fails. The visible symptom was: the warm Alpine+Rust disk seemed reachable but never actually mounted, so cargo and rustc were missing from the user's terminal. This is root cause #2 of issue #3. Probe with mode:'cors' (the default) and return a structured {ok, reason} result. resolveDiskUrl now logs a console.warn — pointing to docs/case-studies/issue-3/analysis-disk-cors.md — when the warm URL is reachable but CORS-blocked, distinguishing it from the benign "asset not built yet" 404 case. Backward compat: the new resolveDiskUrl tolerates legacy probe implementations that return a bare boolean. Tests: 11 new cases in tests/disk-cors.test.mjs covering the cors happy path, TypeError → cors-or-network, 404 → not-found, opaque slip through, wss:// skip, AbortError → timeout, structured warning content, and the legacy-boolean tolerance.

…() (#3) When the page loaded an empty workbench on Pages, the only diagnostic signal was three console errors with no context. Add a tiny opt-in verbose mode so the next regression can be diagnosed in seconds: * createDebug(namespace) — namespaced logger that's a no-op (zero cost) when ?debug is not set; supports ?debug=1 for "all" or ?debug=boot,workbench to filter, plus localStorage.rustWebBoxDebug as a sticky setting. * boot.js logs through it at the four decision points where issue #3's root causes hid: workbench placeholder substitution, CheerpX load, VM phase progress, and disk URL resolution. * __rustWebBox.dump() — JSON-safe runtime snapshot a maintainer can paste into a bug report (href, UA, COI/SAB status, vmPhase, disk URL, subsystem readiness flags). Tests: 11 new cases covering parse precedence (URL > localStorage), no-op behaviour when disabled, namespace filtering, private-mode storage tolerance, and a JSON round-trip of dumpRuntime.

Adds a `--base=/path` flag to `web/build/dev-server.mjs` so the local dev server mirrors how GitHub Pages serves us at `/rust-web-box/`: requests outside the prefix 404, bare-root visits redirect to the prefix, and the prefix is stripped before file lookup. Adds `web/tests/pages-parity.test.mjs` (5 tests) that spawn the dev server in sub-path mode and assert the exact contracts that issue #3 silently broke: * extension manifests reachable under the deploy base * COOP/COEP headers set under the prefix (CheerpX SAB requirement) * inline bootstrap encodes/substitutes `{BASE_PATH}` * outside-prefix paths 404 * bare root redirects to the canonical prefix URL Also commits the after-fix Playwright screenshot to `docs/case-studies/issue-3/screenshots/after-fix.png` showing the workbench mounted with hello_world.rs open, terminal active, and Cargo Run statusbar button visible.

Adds `web/tests/cargo-install.test.mjs` (4 tests) exercising the full sequence of HTTP calls cargo issues for `cargo install serde`: 1. GET https://index.crates.io/config.json (proxy chain) 2. GET https://index.crates.io/se/rd/serde (proxy chain) 3. GET https://crates.io/api/v1/crates/serde (direct, CORS-open) 4. GET https://static.crates.io/crates/serde/...crate (direct) Production routing rules are used unchanged (no test overrides), so this test fails loudly if anyone weakens the host allowlist. Also covers transient proxy failure mid-install — the second proxy must take over without failing the overall install. Closes the R7 acceptance criterion from issue #3.

konard · 2026-04-29T17:23:06Z

Solution summary

All 9 tasks done. PR #4 is ready for review with:

7 commits on issue-3-eb4b77897b15
119 tests passing (24 new across disk-cors, debug-mode, pages-parity, cargo-install)
All CI checks green (9 pass, 0 fail, 6 conditionally skipped)
Visual proof: before/after screenshots in the case study
Three root causes documented and fixed

PR URL: #4

This summary was automatically extracted from the AI working session output.

konard · 2026-04-29T17:23:15Z

🤖 Solution Draft Log

This log file contains the complete execution trace of the AI solution draft process.

💰 Cost estimation:

Public pricing estimate: $13.582026
Calculated by Anthropic: $13.602026
Difference: $0.020000 (+0.15%)

📊 Context and tokens usage:

Claude Opus 4.7: (4 session segments)

115.6K / 1M (12%) input tokens, 7.2K / 128K (6%) output tokens
116.8K / 1M (12%) input tokens, 35.8K / 128K (28%) output tokens
117.0K / 1M (12%) input tokens, 29.5K / 128K (23%) output tokens
108.3K / 1M (11%) input tokens, 18.1K / 128K (14%) output tokens

Total: (444.2K + 16.1M cached) input tokens, 109.7K output tokens, $13.548002 cost

Claude Haiku 4.5:

934 / 64K (1%) output tokens

Total: 24.4K input tokens, 934 output tokens, $0.034024 cost

🤖 Models used:

Tool: Anthropic Claude Code
Requested: opus
Main model: Claude Opus 4.7 (claude-opus-4-7)
Additional models:
- Claude Haiku 4.5 (claude-haiku-4-5-20251001)

📎 Log file uploaded as Gist (5743KB)

View complete solution draft log

Now working session is ended, feel free to review and add any feedback on the solution draft.

konard · 2026-04-29T17:25:38Z

✅ Ready to merge

This pull request is now ready to be merged:

All CI checks have passed
No merge conflicts
No pending changes

Monitored by hive-mind with --auto-restart-until-mergeable flag

This reverts commit 71a1e01.

The live deploy at https://link-foundation.github.io/rust-web-box/ rendered the workbench but CheerpX threw `DataCloneError: SharedArrayBuffer transfer requires self.crossOriginIsolated` on first boot. The header-synthesis half of the fix shipped in PR #4 (`web/sw.js` adds COOP/COEP to subresource responses) but the registration-and-reload half — needed because the *navigation* itself must be intercepted by an already-active SW for isolation to latch — was never landed. Add `web/glue/coi-bootstrap.js`, a classic-script IIFE that runs as the first executable script in `<head>`. It registers `./sw.js` and, when `crossOriginIsolated` is false, forces a one-shot `location.reload()` so the next navigation receives the SW-decorated headers. A `sessionStorage` key (`rust-web-box.coi.reloaded`) guards against reload loops; `?coi=0` is an opt-out for diagnostics. Move SW registration responsibility out of `web/glue/boot.js` so a single owner handles it (boot.js still references the new file in a comment so future readers see where it moved). Tests: `web/tests/coi-bootstrap.test.mjs` covers wiring (script is first in `<head>` of both `index.html` and `build/index.template.html`, classic script with no `defer`/`async`/`type=module`, boot.js no longer calls `serviceWorker.register`, sw.js still synthesizes headers) and behaviour (warm load, ?coi=0 opt-out, no SW API, controller-attached, second-pass loop guard, fresh-load registration with relative URL). All 134 tests in `web/tests/` pass.

Initial commit with task details

71a1e01

Adding .gitkeep for PR creation (default mode). This file will be removed when the task is complete. Issue: #3

konard self-assigned this Apr 29, 2026

konard added 6 commits April 29, 2026 17:03

konard changed the title ~~[WIP] Fix all bugs, and improve quality of our project~~ Fix broken GitHub Pages deploy: substitute {BASE_PATH}, CORS-aware disk probe, debug mode, Pages-parity tests (#3) Apr 29, 2026

konard marked this pull request as ready for review April 29, 2026 17:17

chore(changelog): fragment for issue #3 (Pages deploy fix)

d8ad045

Revert "Initial commit with task details"

6b7d466

This reverts commit 71a1e01.

konard merged commit 1e50810 into main Apr 29, 2026
16 checks passed

konard mentioned this pull request Apr 29, 2026

fix(web): cross-origin isolation bootstrap so CheerpX can boot on Pages (issue #7) #8

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix broken GitHub Pages deploy: substitute {BASE_PATH}, CORS-aware disk probe, debug mode, Pages-parity tests (#3)#4

Fix broken GitHub Pages deploy: substitute {BASE_PATH}, CORS-aware disk probe, debug mode, Pages-parity tests (#3)#4
konard merged 9 commits intomainfrom
issue-3-eb4b77897b15

konard commented Apr 29, 2026 •

edited

Loading

Uh oh!

konard commented Apr 29, 2026

Uh oh!

konard commented Apr 29, 2026

Uh oh!

konard commented Apr 29, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

konard commented Apr 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Root causes (3, ordered by severity)

Acceptance criteria from the issue

Visual proof

Test plan

Reproduction

Case study

Commits

Uh oh!

konard commented Apr 29, 2026

Solution summary

Uh oh!

konard commented Apr 29, 2026

🤖 Solution Draft Log

💰 Cost estimation:

📊 Context and tokens usage:

🤖 Models used:

📎 Log file uploaded as Gist (5743KB)

Uh oh!

konard commented Apr 29, 2026

✅ Ready to merge

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

konard commented Apr 29, 2026 •

edited

Loading