Skip to content

ci: nothing executes the wasm-opt'd bundle under test -- both automated lanes skip wasm-opt, and only deploy/release run it #1019

Description

@bpowers

The gap

src/engine/build.sh runs wasm-opt -O3 in place over core/$out_name, so whatever loads that
blob afterwards loads the optimized one. That means the TypeScript suite executes the optimized
bundle only where wasm-opt actually ran.

It does not run in either automated lane:

caller of pnpm build wasm-opt
.github/workflows/ci.yaml:219 disabled (DISABLE_WASM_OPT=1, deliberate — the comment cites the multi-minute pass)
scripts/pre-commit disabled (170 s per engine-touching commit; see below)
scripts/deploy-web.sh:68 runs
scripts/deploy-web-staged.sh:69 runs
.github/workflows/serve-release.yml:99 runs
.github/workflows/ts-release.yml:74 runs

The four that run it build and ship the bundle; none of them executes the engine test suite
against it. So the optimized bundle is never exercised under test anywhere.

Until the pre-commit change this was covered incidentally, on developer machines only — which is
not coverage that can be relied on: it depended on every developer running the hook, on a machine
with wasm-opt installed, and it was invisible in CI either way.

Why the pre-commit hook stopped doing it

Measured on an engine change that genuinely alters emitted code (a probe that only adds an unused
item is stripped by LTO, leaves the wasm byte-identical, and build.sh's cmp guard then skips
wasm-opt entirely — so it measures a build that never does the expensive part):

  • src/engine/build.sh: 188.7 s, of which wasm-opt is ~170 s (90%)
  • the whole pre-commit hook on such a commit: ~200 s, of which pnpm build is ~90%
  • for comparison, cargo test in the same hook: ~12 s

Cost of running the tests against the unoptimized blob instead: none measurable — engine TS
suite 1.26 / 1.29 s unoptimized vs 1.40 / 1.42 s optimized. The suite is harness-dominated, not
wasm-execution-dominated.

170 s per engine-touching commit is far too much to pay for coverage that only some machines
provide. But the coverage itself is worth having somewhere.

What would close it

A lane that builds with wasm-opt and then runs the engine TypeScript suite against the
resulting blob. It does not need to be on the PR gate — nightly, or path-filtered to engine and
build.sh changes, is enough, because what it guards against is a wasm-opt miscompilation or a
version incompatibility, not a per-PR regression.

Sizing, from the same measurements: the optimizing build is ~170 s more than the plain one, and the
suite it would then run is ~1.3 s.

Related constraint, so a future fix does not take the wrong shape

Do not close this by flipping the default in build.sh or package.json. wasm-opt is off in
two callers and on in four; a default flip routes through the four deploy/release callers and ships
a browser bundle 24% larger (5.00 → 6.20 MB) to every user. The size argument is recorded in
.cargo/config.toml's comment about keeping the wasm target at opt-level="z". The enable/disable
decision belongs to each caller, not to the build script.

Also measured, so it is not re-proposed

Lowering the Rust optimization level to make the wasm build faster is backwards: opt-level=1
emits a 61% larger module (7.90 → 12.71 MB raw) and wasm-opt's cost scales with input size.

profile cargo wasm-opt total
opt-level=z + LTO (today) 34.5 s 170.4 s 205 s
opt-level=1 + LTO 48.4 s 295.5 s 344 s
opt-level=1, no LTO 41.3 s 266.6 s 308 s

opt-level=z is both smaller and faster end to end.

Metadata

Metadata

Assignees

No one assigned

    Labels

    ciCI, build pipeline, test hygiene

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions