Skip to content

fix(reports): every report format is empty under --parallel #1004

Description

@Chemaclass

Summary

--report-junit, --report-tap, --report-json, --report-html and --log-junit all produce a report containing zero tests when the run is parallel. The tests themselves run and pass; only the report is empty.

Reproduced on main and on the released 0.45.0 binary. tests/unit/util is 49 tests:

format sequential --parallel
junit tests="49" tests="0"
tap 1..49 1..0
json "total": 49 "total": 0
html 8452 bytes 1172 bytes (empty shell)
--log-junit tests="49" tests="0"
./bashunit --report-junit /tmp/seq.xml tests/unit/util   # tests="49"
./bashunit --parallel --report-junit /tmp/par.xml tests/unit/util   # tests="0"

Why this matters more than an empty file

--parallel is the fast mode people reach for in CI, and junit is what CI consumes. A junit file reporting zero tests is not obviously broken — many CI systems render it as a green, empty run. So a suite can be fully green, fully reported, and communicating nothing, with no error anywhere to say so.

It is the same shape as #959: parallel losing data silently while the run stays green.

Root cause

bashunit::reports::add_test_passed / _failed / _skipped / _incomplete / _snapshot are called from src/runner/exec.sh (lines ~449-490) and src/runner/hooks.sh. Under --parallel those run inside the forked per-test worker, so the _BASHUNIT_REPORTS_TEST_* arrays they append to die with the worker.

src/state/parallel.sh is what reconstructs per-test results in the parent after the workers finish — and it contains zero references to _BASHUNIT_REPORTS_*:

grep -c 'REPORTS' src/state/parallel.sh   # 0

So the counters survive the fork boundary and the report arrays do not. The writers then serialise an empty array and report zero.

Fix direction

The per-test result payload already crosses the worker boundary (##ASSERTIONS_…##TEST_OUTPUT=…), and state/parallel.sh already parses it to rebuild counters. The report rows need the same treatment: carry the fields add_test_* needs (file, label, status, duration, assertion count, message) through that payload, and replay them in the parent during aggregation.

Worth deciding explicitly: replay into the same add_test_* functions during aggregation, so there is one place that knows the row shape, rather than a second writer that can drift from the first.

Constraints

  • Bash 3.0+; no declare -A, so the rows stay parallel indexed arrays as they are today.
  • The payload is a contract shared by state/payload.sh, runner/payload.sh and helpers.sh — see the encode/decode note in state/payload.sh about keeping both sides byte-identical.
  • Per-test path must stay fork-free — see .claude/rules/perf-fork-budget.md. Aggregation runs once at the end, so cost there is not on the hot path.
  • Failure messages may contain newlines and non-ASCII; the payload already base64-encodes output for this reason.

Acceptance criteria

  • A --parallel run produces the same report contents as the equivalent sequential run, for junit, tap, json and html
  • Test count, names, statuses, durations and failure messages all survive
  • Covered by a test that runs the same suite both ways and compares the reports, so the two cannot drift again
  • Skipped, incomplete and snapshot rows are covered too, not only passed and failed
  • make sa · make lint · ./bashunit --parallel --simple --strict tests/ · bash build.sh bin -v

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

Status
Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions