Skip to content

fix(junit): report suite hook failures via hook.failed event#5657

Open
DavertMik wants to merge 1 commit into
4.xfrom
fix/5645-junit-suite-hook-failures-event
Open

fix(junit): report suite hook failures via hook.failed event#5657
DavertMik wants to merge 1 commit into
4.xfrom
fix/5645-junit-suite-hook-failures-event

Conversation

@DavertMik

Copy link
Copy Markdown
Contributor

Fixes #5645. Alternative to #5656.

The JUnit reporter built XML only from result.tests, which misses failures from BeforeSuite / AfterSuite hooks — those tests never emit event.test.before, so result.tests stays empty and the report shows tests="0" failures="0".

Approach

Instead of tapping into Mocha's runner fail event (as #5656 does), this subscribes to CodeceptJS's own event.hook.failed — the same signal lib/listener/result.js already uses to count failedHooks (visible in the console summary as 2 failedHooks). Failed BeforeSuite/AfterSuite hooks are collected and appended as synthetic failed <testcase> entries before the XML is built. Per-test Before/After hooks are excluded by hookName — those failures already surface through result.tests.

Compared to the runner-tap approach this:

  • needs no container.mocha() access, no attach-timing dance across three events, and no regex parsing of Mocha's English hook titles (/^"(before all|after all)" hook:/)
  • works under run-workers: event.hook.failed is already forwarded from workers to the main process (lib/workers.js:723-724), where the report is written — the runner tap captures nothing there since the main process's mocha never runs
  • works under run-rerun: lib/rerun.js calls mocha.run() without ever assigning mocha.runner, so the runner tap never attaches in that mode at all
  • keeps the JUnit report consistent with the framework's own failedHooks accounting
  • reuses suite.tags computed by enhanceMochaSuite instead of re-parsing the title (the tags regex in fix(junit): report suite hook failures #5656 was also double-escaped and never matched)

The buildXml plumbing (hookFailures param, allTests concat) is the same as in #5656 — credit to @morgan-coded and @mirao for that part and for the investigation on the issue.

Known limitations

  • Under run-workers the forwarded payload (hook.simplify()) doesn't carry the suite, so hook-failure testcases group under a fallback Tests suite instead of the feature title. Fixable later by including the suite title in the worker message.
  • Helper-level _beforeSuite/_afterSuite failures fail through suiteSetup/suiteTeardown, which currently don't emit event.hook.failed — a pre-existing gap (the failedHooks stat misses them too). Firing the event from those two error paths is a small follow-up that would fix every listener at once, not just this plugin.

Verified with

Exact repro from #5645 (throwing BeforeSuite + AfterSuite, junitReporter enabled):

  • codeceptjs run<testsuites tests="2" failures="2"> with both hooks as failed testcases under the correct <testsuite name="My @smoke">, tags property populated, full stack traces (was tests="0" failures="0")
  • codeceptjs run-workers 2 → both hook failures reported (tests="2" failures="2")
  • npx mocha test/unit/junitReporter_test.js — 10 passing
  • npm run lint
  • npm run test:unit — 759 passed
  • npm run test:runner — 275 passed

🤖 Generated with Claude Code

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4.x: junitReporter doesn't record Before/After suite failures

1 participant