Skipped Scenarios missing from cucumber.json files #288

[jcundill]

Addresses #288

Currently skipped tests, those not matching the tags used for the run, are not added to the cucumber.json files generated during that run.

This PR adds them back into the generated files.

Changes:
In createTestsFromFeature pass all the scenarios into createTestFromScenarios but mark the subset that should actually be executed with a shouldRun flag.

In createTestFromScenarios check if the scenario should be executed and if not just add it to the cucumberDataCollector and tell mocha that we skipped this test.

Fix the cucumberDataCollector not to fail the scenario if mocha tells it that the test was skipped.

[lgandecki]

sorry, I took so long, tough few months for our organization... :) Looks good though! Thanks for the work

[lgandecki]

🎉 This PR is included in version 2.4.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀

[badeball]

This has the unforunate consequence of running before- & after-hooks for all skipped tests. If I now mark a test with @focus that is part of a file with 50+ scenarios, then it takes quite a lot of time for Cypress to even reach the focused test. Hence, this breaks the workflow in my case.

I'm not sure what the best option here is though. Generating Cucumber reports is already quite a headache.

[lgandecki]

damn.. that's definitely not what we want. I guess when you use @Focus in the GUI you don't need the reports though? maybe we could disable it unless someone runs cypress with run instead of open?

[badeball]

I'm just brainstorming here, but do we need to call it() at all? Can we skip it, if we merely need to call onStartScenario and onFinishScenario. Omitting it would mess up order I believe and all skipped tests would appear first. Does this matter?

[lgandecki]

I'm not sure.. It sounds reasonable. @jcundill ?

[rcaunt]

We have another issue here now. We pass our output file to Jira TM4J via the automation end point and we get the result Test Result Status with name skipped not found.

We've added this into TM4J and can manually mark our tests cases as skipped within a test cycle but we can't get the automation working with it.

Granted not strictly an issue with this library.

[JuergenKindler]

See the same with Serenity/Cucumber ... seems TM4J endpoint /rest/atm/1.0/automation/execution/cucumber/projectId cannot cope with
"result": { "duration": 2365000, "status": "skipped" }

That seems to be a general problem on TM4J.

[rcaunt]

@JuergenKindler I raised #419 about the issue, it seems that TM4J weren't too fussed about fixing it and therefore I wrote a post-processing script which we run after our tests, prior to submission to TM4J, which removes any tests that are fully skipped.

[JuergenKindler]

That's what I was also planning for now. But as soon as I have a way to contact their support, I will push here.