Test skipped due to failure in previous step

[ylavoie]

In a feature test with examples, failure in one step should only skip remaining steps of the current example.
Currently, a failure in an example skip all other examples.

[pjlsergeant]

I don't think that's accurate - specifically I don't think that's how Cucumber works

…

On Wed, 28 Jun 2017 at 18:09, Yves Lavoie ***@***.***> wrote: In a feature test with examples, failure in one step should only skip remaining steps of the current example. Currently, a failure in an example skip all other examples. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <https://github.com/pjlsergeant/test-bdd-cucumber-perl/issues/123>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABgslR4GEx-cgq9Ibdo-582c7CKyUhVks5sIojVgaJpZM4OIQo3> .

[ylavoie]

Coming soon, see cucumber/common#79

[pjlsergeant]

Heh, this will invalidate most of my MSc dissertation :’) They are proposing something slightly different there, which I like, which is the idea that a step can be marked as being allowed to soft-fail. If you want to implement that, I’ll accept it with open arms. I have a few ideas on how to do that nicely if that’s something you want to do.

…

-P

On Fri, Jun 30, 2017 at 9:23 PM, Yves Lavoie ***@***.***> wrote: Coming soon, see cucumber/common#79 <cucumber/common#79> — You are receiving this because you commented. Reply to this email directly, view it on GitHub <https://github.com/pjlsergeant/test-bdd-cucumber-perl/issues/123#issuecomment-312364694>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AABgskpPtcO0DFA6VjT2M3xDlI9UWlFyks5sJVkogaJpZM4OIQo3> .

[ylavoie]

I do like both. So yes, I'd like to read your ideas about that.
My original goal, and I've seen it shared in many cucumber forums, was to consider examples in outlines as independent scenarios, so the failure of one shouldn't have any impact on others. I will certainly add my voice to the chorus there too.
Soft-fail steps add another nice twist that could give the same effect for outlines and examples.

[ehuelsmann]

It's actually not coming soon. The referenced issue has been closed without resolution.

[ehuelsmann]

I think that if we switch to cucumber/cucumber's gherkin-perl (aka Gherkin module on CPAN) after cucumber/common#694 has been merged, this problem solves itself: In the pickle output each scenario (not scenario outline) is its own testing unit, which means that failure of one "example" won't mean automatic failure of all examples.

[ehuelsmann]

The conclusion on the referred thread is that the community desperately wants it (the soft assertions), but nobody has enough skills to implement it for all backends. So, the issue ended up being closed.

[ehuelsmann]

Just verified in the Cucumber committers Slack channel: each expanded scenario should be treated as a completely separate scenario. The scenario outline is only syntactic sugar to the definition of multiple scenarios.

That means that - as I understand @ylavoie's request - we should implement as he requested.