Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Before Feature hooks are still executed when the feature is ignored. #2234

Closed
9 of 37 tasks
NaZaRKIN123 opened this issue Dec 14, 2020 · 4 comments · Fixed by #2269
Closed
9 of 37 tasks

Before Feature hooks are still executed when the feature is ignored. #2234

NaZaRKIN123 opened this issue Dec 14, 2020 · 4 comments · Fixed by #2269

Comments

@NaZaRKIN123
Copy link

NaZaRKIN123 commented Dec 14, 2020

SpecFlow Version:

  • 3.5
  • 3.4
  • 3.3
  • 3.1
  • 3.0
  • 2.4
  • 2.3
  • 2.2
  • 2.1
  • 2.0
  • 1.9

Used Test Runner

  • SpecFlow+Runner
  • MSTest
  • NUnit
  • Xunit

Version number: 3.12

Project Format of the SpecFlow project

  • Classic project format using packages.config
  • Classic project format using <PackageReference> tags
  • Sdk-style project format

.feature.cs files are generated using

  • SpecFlow.Tools.MsBuild.Generation NuGet package
  • SpecFlowSingleFileGenerator custom tool

Visual Studio Version

  • VS 2019
  • VS 2017

Enable SpecFlowSingleFileGenerator Custom Tool option in Visual Studio extension settings

  • Enabled
  • Disabled

Are the latest Visual Studio updates installed?

  • Yes
  • No, I use Visual Studio version <Major>.<Minor>.<Patch>

.NET Framework:

  • >= .NET 4.5
  • before .NET 4.5
  • .NET Core 2.0
  • .NET Core 2.1
  • .NET Core 2.2
  • .NET Core 3.0
  • .NET Core 3.1
  • .NET 5.0

Test Execution Method:

  • Visual Studio Test Explorer
  • TFS/VSTS/Azure DevOps – Task – PLEASE SPECIFY THE NAME OF THE TASK
  • Command line – PLEASE SPECIFY THE FULL COMMAND LINE

<SpecFlow> Section in app.config or content of specflow.json


Issue Description

Before Feature hooks are still executed when the feature is ignored. I see it by throwing an exception inside Before Feature hook or by debugging these hooks.
It's very annoying as when some tests got outdated and started to fail we usually ignore them until an update is planned, but they still fail.

Steps to Reproduce

  1. Create a new project with 1 feature and 1 test
  2. Create a Before Feature hook and throw an exception inside it
  3. Add Ignore tag and tag created above to a feature file
  4. Run a test.
    Expected: test should be marked as ignored.
    Actual: a test is marked as failed.

Repro Project

@SabotageAndi
Copy link
Contributor

We check if the feature has the @ignore tag and then don't call the feature hooks.
In the case that every scenario is marked with @ignore, we will still call the feature hooks. This behavior needs to be documented at https://github.com/SpecFlowOSS/SpecFlow/blob/master/docs/Bindings/Hooks.md

@NaZaRKIN123
Copy link
Author

Thanks!

@tzongithub tzongithub self-assigned this Jan 15, 2021
@tzongithub tzongithub moved this from Planned to In Progress in OSS Iteration 1 Jan 15, 2021
@tzongithub
Copy link
Contributor

tzongithub commented Jan 18, 2021

I'm experimenting with adding back the test framework specific ignore attributes in the code generation to "simplify the ignore behavior"...
Unfortunately the nUnit tests for ignored features fail with this relatively new error: nunit/nunit3-vs-adapter#811.
The route cause of the issue is tracked here: nunit/nunit#3724.

@github-actions
Copy link

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 12, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
No open projects
Development

Successfully merging a pull request may close this issue.

4 participants