Sub-issue of #235.
Problem
test/test-XX-*/ is advertised as the e2e suite, but several tests are actually in-process unit tests — they require('../../lib-es5/...') directly and assert against pure functions without ever spawning the pkg CLI or producing a binary. Examples found on main:
test-00-sea-picker/main.js — tests pickMatchingHostTargetIndex in lib/sea.ts.
test-48-common/main.js — tests path/snapshot helpers in lib/common.ts.
test-50-config-parse/main.js — tests config + compress-type parsing; uses an ad-hoc in-process t() harness.
Lumping these in with the real e2e suite:
- pays the per-directory process-spawn cost for trivial assertions,
- mixes "lib regression" with "build regression" under one runner,
- makes it hard to add many small, fast unit tests without polluting the e2e list,
- and we have zero coverage visibility today — no
c8, no NODE_V8_COVERAGE, no --experimental-test-coverage anywhere in the repo or CI.
Proposal
Phase 1 — new unit suite on node:test
- Add
test/unit/ with files using the built-in node:test runner. No new runtime deps — we're already on Node >= 22.
- Add scripts:
yarn test:unit → node --test --test-reporter=spec test/unit
yarn test:unit:watch → node --test --watch test/unit
- Migrate the existing unit-in-disguise tests into
test/unit/ as *.test.ts files (loaded via esbuild-register, the same mechanism DEV=true already uses for the CLI):
test-00-sea-picker → test/unit/sea-picker.test.ts
test-48-common → test/unit/common.test.ts
test-50-config-parse → test/unit/config-parse.test.ts
- Remove the old directories once migrated and update any exclusion lists in
test/test.js.
First unit-test targets beyond the migration (pure-ish modules, no filesystem/network):
Phase 2 — coverage across unit + e2e (the tricky part)
The pkg CLI runs as a child process under the e2e suite (utils.pkg.sync → spawn('node', ['lib-es5/bin.js', …])), so a coverage tool that only wraps the test harness sees nothing interesting. Node's NODE_V8_COVERAGE env var is the right primitive: it's honoured by every child process that inherits it — exactly what we need. test/utils.js's utils.pkg.sync already spreads ...process.env into the spawned CLI's env, so the pkg subprocess will emit coverage with zero code changes the moment the var is set upstream.
The open question is what reports over the resulting V8 JSON dump.
Option A — node:test built-in coverage (zero-dep, preferred long-term). node --test --experimental-test-coverage --test-reporter=lcov gathers V8 coverage itself and supports --test-coverage-include=lib/** / --test-coverage-exclude. When enabled, Node sets NODE_V8_COVERAGE internally, so even subprocess coverage is captured. Fits the unit suite perfectly.
The snag is the e2e half: test/test.js is a custom harness, not a node:test run, so the built-in coverage machinery doesn't wrap it. Two sub-options:
- A1. Set
NODE_V8_COVERAGE=coverage/e2e manually on the test.js harness, then stitch together a reporter (roll a ~100 LOC merger, or wait for Phase 3 to migrate e2e onto node:test so the built-in reporter covers everything).
- A2. Defer unified coverage until Phase 3 lands — ship unit coverage alone first.
Option B — c8 as a thin reporter over V8 dumps (pragmatic short-term). Adds c8 as a devDependency; it wraps NODE_V8_COVERAGE, merges per-process coverage, and reports. Scripts:
yarn coverage:unit → c8 --reporter=text --reporter=lcov node --test test/unit
yarn coverage:e2e → c8 --reporter=text --reporter=lcov --clean=false node test/test.js node22 no-npm (--clean=false so unit + e2e accumulate in the same dir).
yarn coverage → runs both, then c8 report over the merged coverage/tmp dir.
Add a .c8rc.json with include: ["lib/**"], exclude: ["lib-es5/**", "test/**", "prelude/**", "**/*.d.ts"], all: true.
Recommendation: start with Option B (c8) because it's the only off-the-shelf merger that bridges the current custom e2e harness to V8 output. Once Phase 3 migrates e2e to node:test, revisit and drop c8 in favour of Option A's built-in reporter. Worst case: we keep c8 long-term — it's ~50 KB, no runtime impact, and does one thing well.
Known caveats (apply to either option):
- SEA binaries embed V8 bytecode; coverage from inside a packaged binary won't map back to
lib/ source. That's fine — we only care about coverage of the build process (lib/bin.ts and its imports), not of the snapshot runtime.
prelude/bootstrap.js runs inside the packaged binary; exclude it from coverage targets.
- Worker-thread bootstrap forks another Node process — it inherits env, so coverage should propagate, but needs a sanity check.
- CI: upload the merged
lcov.info as a workflow artifact. Wiring Codecov / Coveralls is a follow-up, out of scope here.
Phase 3 — stretch: migrate e2e to node:test
Rewrite test/test.js as a node:test entrypoint that discovers test/e2e/**/*.test.js. Benefits: one runner, consistent reporter, built-in --concurrency, and free --test-shard support for CI parallelism (which #241 touches). Each current test-XX-*/main.js becomes a test('name', async (t) => { … }) block. Bonus: unlocks Option A in Phase 2 — built-in coverage across the board, c8 can go away. Non-trivial refactor, so broken out of Phase 1 to keep Phase 1 shippable. If it grows further, split into its own sub-issue.
Acceptance criteria
Sub-issue of #235.
Problem
test/test-XX-*/is advertised as the e2e suite, but several tests are actually in-process unit tests — theyrequire('../../lib-es5/...')directly and assert against pure functions without ever spawning thepkgCLI or producing a binary. Examples found onmain:test-00-sea-picker/main.js— testspickMatchingHostTargetIndexinlib/sea.ts.test-48-common/main.js— tests path/snapshot helpers inlib/common.ts.test-50-config-parse/main.js— tests config + compress-type parsing; uses an ad-hoc in-processt()harness.Lumping these in with the real e2e suite:
c8, noNODE_V8_COVERAGE, no--experimental-test-coverageanywhere in the repo or CI.Proposal
Phase 1 — new unit suite on
node:testtest/unit/with files using the built-innode:testrunner. No new runtime deps — we're already on Node >= 22.yarn test:unit→node --test --test-reporter=spec test/unityarn test:unit:watch→node --test --watch test/unittest/unit/as*.test.tsfiles (loaded viaesbuild-register, the same mechanismDEV=truealready uses for the CLI):test-00-sea-picker→test/unit/sea-picker.test.tstest-48-common→test/unit/common.test.tstest-50-config-parse→test/unit/config-parse.test.tstest/test.js.First unit-test targets beyond the migration (pure-ish modules, no filesystem/network):
lib/common.ts— path normalization, snapshot helpers.lib/detector.ts— AST literal detection (touched often; recently by SEA-ESM: Warning Babel parse has failed: import.meta may appear only with 'sourceType: "module" #264).lib/compress_type.ts— enum + error helpers.lib/help.ts— CLI help string building.Phase 2 — coverage across unit + e2e (the tricky part)
The pkg CLI runs as a child process under the e2e suite (
utils.pkg.sync→spawn('node', ['lib-es5/bin.js', …])), so a coverage tool that only wraps the test harness sees nothing interesting. Node'sNODE_V8_COVERAGEenv var is the right primitive: it's honoured by every child process that inherits it — exactly what we need.test/utils.js'sutils.pkg.syncalready spreads...process.envinto the spawned CLI's env, so the pkg subprocess will emit coverage with zero code changes the moment the var is set upstream.The open question is what reports over the resulting V8 JSON dump.
Option A — node:test built-in coverage (zero-dep, preferred long-term).
node --test --experimental-test-coverage --test-reporter=lcovgathers V8 coverage itself and supports--test-coverage-include=lib/**/--test-coverage-exclude. When enabled, Node setsNODE_V8_COVERAGEinternally, so even subprocess coverage is captured. Fits the unit suite perfectly.The snag is the e2e half:
test/test.jsis a custom harness, not anode:testrun, so the built-in coverage machinery doesn't wrap it. Two sub-options:NODE_V8_COVERAGE=coverage/e2emanually on thetest.jsharness, then stitch together a reporter (roll a ~100 LOC merger, or wait for Phase 3 to migrate e2e ontonode:testso the built-in reporter covers everything).Option B —
c8as a thin reporter over V8 dumps (pragmatic short-term). Addsc8as a devDependency; it wrapsNODE_V8_COVERAGE, merges per-process coverage, and reports. Scripts:yarn coverage:unit→c8 --reporter=text --reporter=lcov node --test test/unityarn coverage:e2e→c8 --reporter=text --reporter=lcov --clean=false node test/test.js node22 no-npm(--clean=falseso unit + e2e accumulate in the same dir).yarn coverage→ runs both, thenc8 reportover the mergedcoverage/tmpdir.Add a
.c8rc.jsonwithinclude: ["lib/**"],exclude: ["lib-es5/**", "test/**", "prelude/**", "**/*.d.ts"],all: true.Recommendation: start with Option B (c8) because it's the only off-the-shelf merger that bridges the current custom e2e harness to V8 output. Once Phase 3 migrates e2e to
node:test, revisit and drop c8 in favour of Option A's built-in reporter. Worst case: we keep c8 long-term — it's ~50 KB, no runtime impact, and does one thing well.Known caveats (apply to either option):
lib/source. That's fine — we only care about coverage of the build process (lib/bin.tsand its imports), not of the snapshot runtime.prelude/bootstrap.jsruns inside the packaged binary; exclude it from coverage targets.lcov.infoas a workflow artifact. Wiring Codecov / Coveralls is a follow-up, out of scope here.Phase 3 — stretch: migrate e2e to
node:testRewrite
test/test.jsas anode:testentrypoint that discoverstest/e2e/**/*.test.js. Benefits: one runner, consistent reporter, built-in--concurrency, and free--test-shardsupport for CI parallelism (which #241 touches). Each currenttest-XX-*/main.jsbecomes atest('name', async (t) => { … })block. Bonus: unlocks Option A in Phase 2 — built-in coverage across the board, c8 can go away. Non-trivial refactor, so broken out of Phase 1 to keep Phase 1 shippable. If it grows further, split into its own sub-issue.Acceptance criteria
yarn test:unitruns the unit suite onnode:test; no new runtime deps.test-XX-*/directories removed.yarn coverageproduces a merged lcov report covering both unit and e2e runs.lib/is recorded in the implementing PR's description so we can see it grow over time.node:test, or a separate sub-issue filed to track it. When this lands, revisit Phase 2 and dropc8in favour of the built-in coverage reporter.