fix(junit): report suite hook failures via hook.failed event#5657
Open
DavertMik wants to merge 1 commit into
Open
fix(junit): report suite hook failures via hook.failed event#5657DavertMik wants to merge 1 commit into
DavertMik wants to merge 1 commit into
Conversation
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #5645. Alternative to #5656.
The JUnit reporter built XML only from
result.tests, which misses failures fromBeforeSuite/AfterSuitehooks — those tests never emitevent.test.before, soresult.testsstays empty and the report showstests="0" failures="0".Approach
Instead of tapping into Mocha's runner
failevent (as #5656 does), this subscribes to CodeceptJS's ownevent.hook.failed— the same signallib/listener/result.jsalready uses to countfailedHooks(visible in the console summary as2 failedHooks). FailedBeforeSuite/AfterSuitehooks are collected and appended as synthetic failed<testcase>entries before the XML is built. Per-testBefore/Afterhooks are excluded byhookName— those failures already surface throughresult.tests.Compared to the runner-tap approach this:
container.mocha()access, no attach-timing dance across three events, and no regex parsing of Mocha's English hook titles (/^"(before all|after all)" hook:/)run-workers:event.hook.failedis already forwarded from workers to the main process (lib/workers.js:723-724), where the report is written — the runner tap captures nothing there since the main process's mocha never runsrun-rerun:lib/rerun.jscallsmocha.run()without ever assigningmocha.runner, so the runner tap never attaches in that mode at allfailedHooksaccountingsuite.tagscomputed byenhanceMochaSuiteinstead of re-parsing the title (the tags regex in fix(junit): report suite hook failures #5656 was also double-escaped and never matched)The
buildXmlplumbing (hookFailuresparam,allTestsconcat) is the same as in #5656 — credit to @morgan-coded and @mirao for that part and for the investigation on the issue.Known limitations
run-workersthe forwarded payload (hook.simplify()) doesn't carry the suite, so hook-failure testcases group under a fallbackTestssuite instead of the feature title. Fixable later by including the suite title in the worker message._beforeSuite/_afterSuitefailures fail throughsuiteSetup/suiteTeardown, which currently don't emitevent.hook.failed— a pre-existing gap (thefailedHooksstat misses them too). Firing the event from those two error paths is a small follow-up that would fix every listener at once, not just this plugin.Verified with
Exact repro from #5645 (throwing
BeforeSuite+AfterSuite, junitReporter enabled):codeceptjs run→<testsuites tests="2" failures="2">with both hooks as failed testcases under the correct<testsuite name="My @smoke">, tags property populated, full stack traces (wastests="0" failures="0")codeceptjs run-workers 2→ both hook failures reported (tests="2" failures="2")npx mocha test/unit/junitReporter_test.js— 10 passingnpm run lintnpm run test:unit— 759 passednpm run test:runner— 275 passed🤖 Generated with Claude Code