Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
fail in AfterStep leaves step successful but scenario failed #1101
Results in all steps being indicated as passing.
When I run a scenario
When I run a scenario
Steps to Reproduce (for bugs)
Context & Motivation
Visually, especially if it is the last step in the scenario, it is quite confusing to see all steps passed but the scenario fails.
I can understand why it happens, but it does seem surprising. The problem is that the way we currently model it, the after-step hook is just another test step in sequence after the gherkin step it was "hooked" onto. So that original gherkin step has already run and passed.
I've been thinking for a while that it would help us to be able to model "nested" steps or sub-steps, as this would also help us get rid of a lot of the edge-cases with around hooks. But this is just vague blue-sky thinking, I don't have anything concrete for how we would do it.
In this case, I wonder if it would be better for now if we were more transparent about the hook steps that are also running, and showing their pass / fail status (or at least their fail status).
Maybe we could include hooks in the summary counts?
I doubt that after-step hooks were introduced to do extra verification in addition to the verification in the step definitions, so from that perspective it is not expected behaviour that an failure in an after-step hook fails the step it is after-step hook for.
Comparing with scenarios and their before and after hooks, we consider both the results of each step definition and hook individually, and combine their results to the result for the scenario (the individual results are clearly documented in the json formatter output). To let an after-step hook fail the step would then mean that we both consider the result of the step definition and the after-step hook(s) individually, and also calculate a result for the step and after-step hook combo. This looks like a major change of the implemented execution model we have today, I don't think it should be considered before Cucumber-Ruby version 4.