Suggestion: Add Option to Run After Hooks Despite Exceptions

[AndyLPK247]

It is possible to have multiple after hooks, and the order of execution may be controlled with the Order parameter. However, if an exception is thrown from an after hook method, then no subsequent after hooks of the same type are executed. This is problematic for my team because we do web testing using Selenium WebDriver. We rely upon after hooks to properly clean up our web tests. It is also arguably unintuitive - my original presumption was that after hooks would always be run for safety purposes.

Consider the following example:

1. AfterScenario hook Syntax validation for feature files in Visual Studio #1 - run for all scenarios - capture a screen shot
2. AfterScenario hook Intellisense for feature files in Visual Studio  #2 - run for specifically tagged scenarios - do some test-specific cleanup (like page navigation or deleting records)
3. AfterScenario hook Create step binding skeletion from Visual Studio #3 - run for all scenarios - log out and possibly dispose the web driver

If hook #2 throws an exception, then hook #3 is not run.

There are a few workarounds:

• I can combine all AfterScenario hooks into one. However, that is not a good solution for scalable development. Every custom case would need to be shoved into one method. Having separate hooks with careful ordering is definitely preferred.
• I can wrap every AfterScenario hook in a "try { } catch (Exception) { }" block. This isn't pretty, but it would guarantee that the final cleanup hook would get run.

What would be really nice is if After hooks had an extra attribute parameter called "AlwaysRun" or "Guaranteed" or something similar that would make the hook run even if previous hooks aborted with exceptions. I could add such a parameter to hook #3 and not need any of the aforementioned workarounds. This sort of solution would be applicable for all after hook types: step, scenario, scenario block, and test run.

[SabotageAndi]

@gasparnagy We had in the past a discussion about the topic of exceptions in hooks, but I don't find it anymore.
I wouldn't change the current behaviour, because this would be a breaking change for users, which would be silent for them (the test is suddenly green after the update, where before there was an error).
Putting everywhere try-catch blocks is not nice in the binding code.

But somehow I have a little aversion against the idea with the additional attribute. Doesn't feel right somehow for me.
If we do this, I would name the attribute "ContinueOnError".

[AndyLPK247]

@SabotageAndi - I like your proposed parameter name "ContinueOnError" better than the ones I suggested.

Also, at the very least, could the current behavior of after hooks please be documented on the SpecFlow doc pages? The current Hooks page does not indicate that if an exception happens in an after hook, then subsequent after hooks of the same type will not be called. It would be very helpful to make this explicitly stated to avoid confusion. Users could presume either the current behavior or (like I did) that subsequent after hooks would be run despite exceptions. https://github.com/techtalk/SpecFlow/wiki/Hooks

[SabotageAndi]

@AndyLPK247 you are right, this information is missing in the documentation
@Stephen-Mc could you add it please? Thanks!

[Stephen-Mc]

I've added a note to the hooks section under "Hook Execution Order". This behaviour applies to both before and after hooks. Here's the change:

Note: If a hook throws an unhandled exception, subsequent hooks of the same type are not executed. If you want to ensure that all hooks of the same types are executed, you need to handle your exceptions manually.

[AndyLPK247]

Awesome! Thanks.

…

On Tue, Dec 5, 2017 at 5:50 AM, Stephen McCafferty ***@***.*** > wrote: I've added a note to the hooks section under "Hook Execution Order". This behaviour applies to both before and after hooks. Here's the change: Note: If a hook throws an unhandled exception, subsequent hooks of the same type are not executed. If you want to ensure that all hooks of the same types are executed, you need to handle your exceptions manually. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#978 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ARJ9UtjajuGtk4E63SNBJpS5vDe3tkaiks5s9R_egaJpZM4Q00u8> .

[goncharr]

Hi @SabotageAndi , In my case I'm using an attribute "AfterTestRun" and after that running DB Hook (cleaning test data) - is it possible to add some functionality which helps to skip running DB Hook if some of the tests are failed. I've tried to use ScenarioContext.Current but unfortunately it didn't work as this Context saved only latest scenario.

[SabotageAndi]

To make it easier for you all to vote on feature requests and influence our prioritization of them in our backlog, we created in our new support tool a special forum to have a better overview of all of them. We moved this feature request there and you can find it here. Sadly we could not take over the existing votes. So if you are still interested in this feature request, please vote again on the new list.

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.