Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: report considers skipped tests as failed #133

Open
dialex opened this issue Aug 1, 2019 · 2 comments
Open

Bug: report considers skipped tests as failed #133

dialex opened this issue Aug 1, 2019 · 2 comments

Comments

@dialex
Copy link

dialex commented Aug 1, 2019

I'm using your report with Cypress. I have one skipped Cypress test:

Screenshot 2019-07-25 at 14 39 35

Expected: either 100% tests passed (green); or 93% passed and 7% tests skipped (grey)
Actual: bar shows red and 7% of tests failed, which is not true

It looks to me your report considers skipped tests as failed

@adamgruber
Copy link
Owner

A skipped test is usually the result of a failed before hook, which the report does consider.

@dialex
Copy link
Author

dialex commented Aug 2, 2019

There's a related ongoing discussion on Cypress cypress-io/cypress#3092 (comment)

This was not the case, I skipped the test deliberately, not because a hook failed. I would prefer the reporter wouldn't assume things. If the test was skipped, show it as skipped. Simple and transparent.

Because right now you have incoherent data. Your report says at the same time 0% and 7% tests failed. The testing world is all about information and confidence, and incoherent data destroys any confidence in it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants