Skipping tests generates screenshots as if the tests failed

[redReno]

Current behavior

I use --env TAGS to skip scenarios in my test suite, which I run headlessly. The test run passes but screenshots are still generated for each scenario that was skipped, as if it failed. Using cy.skip in a vanilla test suite works as intended.

Desired behavior

Skipping scenarios by filtering them with the TAGS environment variable should not generate screenshots.

Test code to reproduce

https://github.com/redReno/cypress-cucumber-examples/tree/main

Clone the repo, run yarn. Then:
yarn cypress_fail to run the tests and generate screenshots. Uses Cypress 8.4.0
yarn cypress_pass to run the tests without generating screenshots. Uses Cypress 8.3.1

Versions

I am using a Debian environment.

• Cypress version:
  I see the issue in all Cypress versions starting with 8.4.0.
  8.3.1 works fine

• Preprocessor version:
  4.3.0 but I have seen the same behavior in older versions.

• Node version:
  16.1.0

[j0meinaster]

Thanks for reporting. Experienced the same issue.

[Thairam]

Me too @redReno @j0meinaster I hope this issue is resolved quickly.

[dholmes2409]

We're having the same issue, did any of you manage to fix this?

[redReno]

Cypress 9.2.0 has a change that might fix this issue. I will upgrade and test it out.

[redReno]

UPDATE: I updated to Cypress 9.3.1 and the issue seems to have been resolved, at least for me.

[dholmes2409]

UPDATE: I updated to Cypress 9.3.1 and the issue seems to have been resolved, at least for me.

Can confirm, works for me too

[badeball]

Due to personal reasons, the previous maintainers of this package are stepping down and handing the reigns over to me, a long-time contributor to the project and a user of it myself. This is a responsibility I'm very excited about. Furthermore, I'd like to thank @lgandecki ++ for all the work that they've done so far.

Read more about the transfer of ownership here.

The repository has however moved and all outstanding issues are being closed. This is not a reflection of the perceived importance of your reported issue. However, if after upgrading to the new version, you still find there to be an issue, feel free to open up another ticket or comment below. Please make sure to read CONTRIBUTING.md before doing so.