Skip to content

CLI compat (phase E): byte-for-byte junit + Rust unit tests + JS integration tests#587

Merged
bokuweb merged 2 commits intootelfrom
cli-compat-phase-e
Apr 18, 2026
Merged

CLI compat (phase E): byte-for-byte junit + Rust unit tests + JS integration tests#587
bokuweb merged 2 commits intootelfrom
cli-compat-phase-e

Conversation

@bokuweb
Copy link
Copy Markdown
Member

@bokuweb bokuweb commented Apr 18, 2026

Summary

Follow-up to merged #586.
Fixes a junit-format compat bug that snuck into phase D and adds a real
test suite (Rust unit + JS integration) so we can keep catching drift
like this before release.

Fix: junit XML is now byte-identical to classic reg-cli

Phase D shipped a simplified JUnit schema (<testsuites> bare,
classname="reg-cli", message="changed|new|deleted") that diverged
from what classic reg-cli produces via xmlbuilder2 — which is exactly
what downstream CI parsers and test/cli.test.mjs snapshot-check.

Corrected to the classic schema:

  • <?xml version="1.0"?> (no encoding attr)
  • <testsuites name="reg-cli tests" tests=N failures=M> (attrs on both
    testsuites and testsuite)
  • <testcase name="..."> (no classname)
  • <failure message="failed"/> for failed items; "newItem" /
    "deletedItem" only when -E/--extendedErrors is set, otherwise
    new/deleted are counted as passed testcases (classic behaviour)

To make that last branch possible, extended_errors is now plumbed through
Optionsreg_cli clap (-E/--extendedErrors) → compare() library
forwarding → build_junit_xml. The JS CLI wrapper still owns exit-code
semantics for -E, but the flag is now dual-forwarded so Rust can also
see it.

New tests

Rustcargo test -p reg_core --lib, 6 new cases in
crates/reg_core/src/report.rs:

  • single failure
  • passed + failed mix
  • new/deleted with and without extended_errors (both branches)
  • XML attribute escaping (&, <, >, ")
  • empty report → self-closing <testsuite/>

JSnode --test, 19 tests across two files (no new deps):

  • js/test/cli.test.mjs (13) — spawns the built CLI, asserts exit codes
    (-I, -E), reg.json schema keys, JUnit XML byte-for-byte vs
    classic, -R, -X client, -F (verifies source reg.json is immutable
    and no diff dir is recreated), stdout formatting, -D custom trailer.
  • js/test/library.test.mjs (6) — compare() EventEmitter lifecycle
    (startcompare×Ncomplete), junit/reg.json via Rust,
    update: true file copy + update event,
    additionalDetection: 'client', and the legacy
    enableClientAdditionalDetection: true alias.

CI

New wasm-test job runs cargo test -p reg_core --lib, then builds the
js dist (using the committed js/reg.wasm — no wasi-sdk needed) and runs
pnpm --filter ./js test. The classic test job is unchanged.

Notable compat finding

Documented in the library test header: wasm32-wasip1-threads libstd's
prestat enumeration only honours the first path segment of a preopen
name. computeWasiSandbox collapses every touched dir into a single
common ancestor, so when all paths live under one scratch (update mode),
that scratch must itself be one segment deep at the repo root. The
lib-test scaffolding uses .libtest-<pid>-<n>-<rand> at the repo root
for exactly this reason. This is a pre-existing limitation worth a
follow-up (multi-segment preopen support) — not something introduced by
this PR.

Test plan

  • cargo test -p reg_core --lib — 10/10 pass.
  • bash scripts/build-wasm.sh + pnpm --filter ./js build — clean.
  • pnpm --filter ./js test — 19/19 pass (13 CLI + 6 library).
  • JUnit XML byte-equal to classic reg-cli's xmlbuilder2 output.
  • -F regenerates HTML/JUnit without creating diff/; reg.json
    unchanged afterwards.
  • -X client flips ximgdiffConfig.enabled in HTML.
  • CI green on otel base.

🤖 Generated with Claude Code

bokuweb and others added 2 commits April 18, 2026 22:26
…gration tests

## Fix: junit XML is now byte-identical to classic reg-cli

Phase D (#586) shipped a simplified junit shape (`<testsuites>` bare,
`classname="reg-cli"`, `message="changed|new|deleted"`) that diverged
from what classic reg-cli produces via xmlbuilder2 — which is exactly
what downstream CI parsers and `test/cli.test.mjs` snapshot-check.

Fixed to the classic schema:

- `<?xml version="1.0"?>` (no `encoding` attr)
- `<testsuites name="reg-cli tests" tests=N failures=M>` (attrs on BOTH
  testsuites and testsuite)
- `<testcase name="...">` (no `classname`)
- `<failure message="failed"/>` for failedItems, `"newItem"`/`"deletedItem"`
  for new/deleted ONLY when `-E/--extendedErrors` is set
- Without `-E`, new/deleted items are counted as passed testcases (classic
  behaviour)

To make that last branch possible, `extended_errors` is now plumbed through
`Options` → `reg_cli` clap (`-E/--extendedErrors`) → `compare()` library
forwarding → `build_junit_xml`. The JS CLI wrapper still owns exit-code
semantics for `-E`, but the flag is now dual-forwarded so Rust can also see
it.

## New tests

Rust (`cargo test -p reg_core --lib`, 6 new tests):
- single failure
- passed + failed mix
- new/deleted with and without extended_errors (both branches)
- XML attribute escaping (&, <, >, ")
- empty report → self-closing `<testsuite/>`

JS (`node --test`, 19 tests across two files, no new deps):
- `js/test/cli.test.mjs` — spawns `node dist/cli.mjs`, asserts exit codes
  (`-I`, `-E`), reg.json schema, JUnit XML **byte-for-byte match** to
  classic, `-R`, `-X client`, `-F` (regenerate from reg.json, verifies
  source reg.json is immutable and no diff dir is recreated), stdout
  formatting, `-D` custom trailer.
- `js/test/library.test.mjs` — `compare()` EventEmitter lifecycle
  (`start`→`compare(xN)`→`complete`), JUnit + reg.json via Rust, `update: true`
  file copy + `update` event, `additionalDetection: 'client'` and the
  legacy `enableClientAdditionalDetection: true` alias.

Notable compatibility finding documented in the library test header:
`wasm32-wasip1-threads` libstd's prestat enumeration only honours the
first path segment of a preopen name. `computeWasiSandbox` collapses
every touched dir to a single common ancestor, so when all positional
dirs live under one scratch (update mode), that scratch must itself be
one segment deep at the repo root. The library test scaffolding uses
`.libtest-<pid>-<n>-<rand>` for exactly that reason.

## CI

New `wasm-test` job runs `cargo test -p reg_core --lib`, then builds the
js dist (using the committed `js/reg.wasm`, no wasi-sdk needed) and runs
`pnpm --filter ./js test`. The classic `test` job is unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`reg_core` embeds the report UI bundle via include_str!:

    let js = include_str!("../../../report/ui/dist/report.js");
    let css = include_str!("../../../report/ui/dist/style.css");

So `cargo test -p reg_core --lib` fails at macro expansion unless
`report/ui/dist/` has been populated. Run `scripts/build-ui.sh v0.3.0`
(same version the root `build:report` npm script uses) before the cargo
test step, and enable corepack so the yarn that script invokes is
available.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@bokuweb bokuweb merged commit 609edad into otel Apr 18, 2026
4 checks passed
@bokuweb bokuweb deleted the cli-compat-phase-e branch April 18, 2026 13:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant