[🐛 Bug]: @wdio/junit-reporter - Both Errors and Failures Incremented for Failed Tests

[ASaiAnudeep]

Have you read the Contributing Guidelines on issues?

• I have read the Contributing Guidelines on issues.

WebdriverIO Version

8.32.4

Node.js Version

20

Mode

WDIO Testrunner

Which capabilities are you using?

NA

What happened?

The JUnit report incorrectly increments both the "errors" and "failures" counts when a test case fails. Seems, failures and errors are being treated similarly - link

What is your expected behavior?

Only the "failures" count should increase for failed tests. JUnit reports typically distinguish between errors (unexpected exceptions) and failures (asserted conditions not met).

JUnit Schema Links

• https://github.com/windyroad/JUnit-Schema/blob/master/JUnit.xsd
• https://github.com/testmoapp/junitxml

How to reproduce the bug.

<testsuite name="As a user I can log into the secure area" timestamp="2023-11-01T21:55:07" time="2.554" tests="3" failures="3" errors="3" skipped="0">

Test Case Expected Output Reference

Relevant log output

NA

Code of Conduct

• I agree to follow this project's Code of Conduct

Is there an existing issue for this?

• I have searched the existing issues

[christian-bromann]

JUnit reports typically distinguish between errors (unexpected exceptions) and failures (asserted conditions not met).

I found the responsible code path here:

webdriverio/packages/wdio-junit-reporter/src/index.ts

Lines 195 to 213 in 146668c

	if (test.error) {
	if (test.error.message){
	test.error.message = test.error.message.replace(ansiRegex, '')
	}
	if (this.options.errorOptions) {
	const errorOptions = this.options.errorOptions
	for (const key of Object.keys(errorOptions)) {
	testCase[key]((test.error as any)[errorOptions[key]])
	}
	} else {
	// default
	testCase.error(test.error.message)
	}
	testCase.standardError(`\n${test.error.stack?.replace(ansiRegex, '')}\n`)
	} else {
	testCase.error()
	}
	testCase.failure()

If we can only report failures, and are able to pass in a failure message, I am happy to accept a pull request with necessary changes.

All contributions are welcome.

[wdio-bot]

Thanks for reporting!

We greatly appreciate any contributions that help resolve the bug. While we understand that active contributors have their own priorities, we kindly request your assistance if you rely on this bug being fixed. We encourage you to take a look at our contribution guidelines or join our friendly Discord development server, where you can ask any questions you may have. Thank you for your support, and cheers!