Skip to content

conformance-ct: gate wall time — sequential targets, per-run jco transpile, over-broad CI cache key #307

Description

@lann

Three mechanical cost items on the migrated gate. Measured locally (warm, no browser; 16,427 -> 18,022 shared cases across the comparison):

incumbent (7110a99) conformance-ct (main)
wasmtime target 9.4s 5.5s
composed target 19.7s 27.9s
jco-node target 40.2s 45.7s
full local gate 42.7s 70.7s

The engine is fine (ct-driver's --jobs 8 halves the wasmtime leg); the topology regressed:

  1. Sequential targets. all: lock-check class-d run-wasmtime run-composed run-jco ... serializes what the incumbent ran through scripts/parallel-recipes.sh; the old full gate was bounded by its slowest target. Restoring target-level parallelism returns most of the +66%.
  2. Per-run transpile. run-jco runs npm install --silent && npm run transpile && npm run transpile:signing on every invocation, even when the guest artifacts are unchanged; the incumbent kept transpiles in the build phase. Split them into a dependency recipe keyed on the artifacts.
  3. CI artifact-cache key over-invalidates. The key hashes conformance/** including prose and outputs (README.md, the committed matrices), so documentation-only and matrix-refresh commits repay the full ~8min build (conformance: ratify the results-schema tolerance note (S5) #301, a README-only change, ran 8m09s/8m19s). In an 11-run post-merge sample the cache never hit once on main. Narrow the key to build inputs (exclude *.md), and note the residual: the key must keep covering everything the build reads (it currently does; a future flag moved into an unhashed justfile would stale it silently).

CI context: pre-migration total workflow wall was ~6-7min with conformance embedded in rust-checks; post-migration rust-checks improved to ~2.5-3min but the conformance job is the critical path at 7-16.5min in every sampled run.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions