Specflow Plugin Failure VS2015 and VS2017

[kdjjoshi]

Specflow Plugin Failure VS2015 and VS2017

SpecFlow Version:

• 2.1

Used Test Runner

• SpecFlow+Runner
• MSTest

Visual Studio Version

• VS 2017
• VS 2015

Are the latest Visual Studio updates installed?

• Yes

.NET Framework:

• >= .NET 4

Test Execution Method:

• Visual Studio Test Explorer
• TFS/VSTS – Task – PLEASE SPECIFY THE NAME OF THE TASK

<SpecFlow> Error in the .cs StepDefinition File

System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.InvalidOperationException: Collection was modified; enumeration operation may not execute.
   at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
   at System.Collections.Generic.Dictionary`2.Enumerator.MoveNext()
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at BoDi.ObjectContainer.NamedInstanceDictionaryRegistration.Resolve(ObjectContainer container, RegistrationKey keyToResolve, ResolutionList resolutionPath)
   at BoDi.ObjectContainer.ResolveObject(RegistrationKey keyToResolve, ResolutionList resolutionPath)
   at BoDi.ObjectContainer.Resolve(Type typeToResolve, ResolutionList resolutionPath, String name)
   at BoDi.ObjectContainer.Resolve[T](String name)
   at BoDi.ObjectContainer.Resolve[T]()
   at TechTalk.SpecFlow.Generator.UnitTestConverter.FeatureGeneratorRegistry..ctor(IObjectContainer objectContainer)
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
   at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at BoDi.ObjectContainer.CreateObject(Type type, ResolutionList resolutionPath, RegistrationKey keyToResolve)
   at BoDi.ObjectContainer.TypeRegistration.Resolve(ObjectContainer container, RegistrationKey keyToResolve, ResolutionList resolutionPath)
   at BoDi.ObjectContainer.ResolveObject(RegistrationKey keyToResolve, ResolutionList resolutionPath)
   at BoDi.ObjectContainer.Resolve(Type typeToResolve, ResolutionList resolutionPath, String name)
   at BoDi.ObjectContainer.<>c__DisplayClass55_0.<ResolveArguments>b__0(ParameterInfo p)
   at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)
   at BoDi.ObjectContainer.ResolveArguments(IEnumerable`1 parameters, RegistrationKey keyToResolve, ResolutionList resolutionPath)
   at BoDi.ObjectContainer.CreateObject(Type type, ResolutionList resolutionPath, RegistrationKey keyToResolve)
   at BoDi.ObjectContainer.TypeRegistration.Resolve(ObjectContainer container, RegistrationKey keyToResolve, ResolutionList resolutionPath)
   at BoDi.ObjectContainer.ResolveObject(RegistrationKey keyToResolve, ResolutionList resolutionPath)
   at BoDi.ObjectContainer.Resolve(Type typeToResolve, ResolutionList resolutionPath, String name)
   at BoDi.ObjectContainer.Resolve[T](String name)
   at BoDi.ObjectContainer.Resolve[T]()
   at TechTalk.SpecFlow.Generator.TestGeneratorFactory.CreateGenerator(ProjectSettings projectSettings)
   at TechTalk.SpecFlow.VisualStudio.CodeBehindGenerator.Actions.GenerateTestFileAction.GenerateTestFile(GenerateTestFileParameters opts)

Repro Project

Issue Description

I have both VS 2015 Update 3 and VS 2017 15.7 installed on my machine. After installing new VS2017 I downloaded the specflow for visual studio 2017 to install in VS 2017 Enterprise (15.7) I am able to build the code after I do getlatest but not when I edit. When I try to Add New Item a new feature file I am not able to do this using VS 2017 I can't see the SpecFlow feature template, or, indeed, any SpecFlow template. I get following error when I am trying to build after I edit/Add a line in the feature file - Above is the error in .cs file for respective feature file. Others in my team are able to build the latest code as well as they are able to edit feature file and build it successfully. How do I fix this issue on my local? I don't see any other updated version of SpecFlow where this issue got fixed. Please advise.

[kdjjoshi]

Regenerating feature files did not help

[SabotageAndi]

As I understand it, you have the VS Extension installed, but you don't have the item templates.
Do you have the SpecFlow options under Tools\Options?
This:

Generally I would suggest to update SpecFlow. 2.1 is really old (2 years) and to use the MSBuild Generation (https://specflow.org/documentation/Generate-Tests-from-MsBuild/)

[kdjjoshi]

It seems this issue started coming when SpecFlow Extension for VS2015 got updated on 07/10/2018.

General setting for SpecFlow, 'Disable re-generate feature file popup' is "False"

@SabotageAndi

[SabotageAndi]

@kdjjoshi With 2017.2.1 we changed the default Generation Mode to "OutOfProcess". Before it was "AppDomain".
You can try to change it back and see if it works.

But be aware that we will remove the AppDomain generation mode in a future release of the VS Extension.
So I suggest you plan some time to upgrade to the latest version of SpecFlow and use the MSBuild generation mode (which will be default in SpecFlow 3).

[kdjjoshi]

@SabotageAndi changing setting to "AppDomain" did not work with having 2017.2.1 specflow extension installed. Fails with below error-

Severity	Code	Description	Project	File	Line	Suppression State
Error		Custom tool error: Generation error: Exception has been thrown by the target of an invocation. -> Could not load type 'TechTalk.SpecFlow.Configuration.SpecFlowConfiguration' from assembly 'TechTalk.SpecFlow, Version=2.1.0.0, Culture=neutral, PublicKeyToken=0778194805d6db41'.	PerformHTML	C:\HTML\FeatureFiles\HTML_Regression.feature	2	

Also tried this using this setting with SpecFlow 2.3 version. It still fails. Is there any plan of fixing this issue happening for SpecFlow and MSTest builds with this version of extension?

[SabotageAndi]

@kdjjoshi So you get this error with SpecFlow 2.3.2 and AppDomain or OutOfProcess generation mode?
That's new.

Can you create a project for me to reproduce this error?

You can find older versions of the VS Extension here: https://github.com/techtalk/SpecFlow.VisualStudio/releases
But be aware that the VS Autoupdater need to be deactivated for the SpecFlow extension.

[kdjjoshi]

@SabotageAndi I am using solution with SpecFlow 2.1 , however this issue was reproducible when I upgraded SpecFlow to 2.3.1 only after adding extension 2017.2.1.
Yes, I am making sure autoupdater is off whenever I am installing any older versions of this extension.

[kdjjoshi]

@SabotageAndi Thank you for helpful info. Due to references you mentioned was able to go through other articles related to similar issues.

Further more, downgrading SpecFlow Extension to TechTalk.SpecFlow.Vs2015Integration.2017.1.10 (most difficult part) in VS2015 with SpecFlow 2.1 fixed above error.

Note - Please forcefully disable updated for SpecFlow extension which gets automatically updated after restart.
To Do so- Tools/Options/Environment/ExtensionsAndUpdates/uncheck all except admin related setting

Since I am no more receiving error while building the project now. Was successfully able to add/edit a test case from feature file and able to build project successfully now.

However, when I am running the the test case. It is throwing below error after I execute TC from test explorer. (still on SpecFlow 2.1)

-- Run test started ------
System.NullReferenceException: Object reference not set to an instance of an object.
   at PerformAutomation.Hooks.AfterTestRun() in e:\Builds\22\ Dev\Automation.UI.Shared\src\Shared\Utilities\Hooks.cs:line 232
   at lambda_method(Closure , IContextManager )
   at TechTalk.SpecFlow.Bindings.BindingInvoker.InvokeBinding(IBinding binding, IContextManager contextManager, Object[] arguments, ITestTracer testTracer, TimeSpan& duration)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.InvokeHook(IBindingInvoker invoker, IHookBinding hookBinding, HookType hookType)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.FireEvents(HookType hookType)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.OnTestRunEnd()
   at TechTalk.SpecFlow.TestRunner.OnTestRunEnd()
   at TechTalk.SpecFlow.TestRunnerManager.OnTestRunnerEnd()
   at TechTalk.SpecFlow.TestRunnerManager.<InitializeBindingRegistry>b__19_0(Object <p0>, EventArgs <p1>)
   at PerformAutomation.Hooks.AfterTestRun() in e:\Builds\22\ Dev\ .Automation.UI.Shared\src\Shared\Utilities\Hooks.cs:line 232
   at lambda_method(Closure , IContextManager )
   at TechTalk.SpecFlow.Bindings.BindingInvoker.InvokeBinding(IBinding binding, IContextManager contextManager, Object[] arguments, ITestTracer testTracer, TimeSpan& duration)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.InvokeHook(IBindingInvoker invoker, IHookBinding hookBinding, HookType hookType)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.FireEvents(HookType hookType)
   at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.OnTestRunEnd()
   at TechTalk.SpecFlow.TestRunner.OnTestRunEnd()
   at TechTalk.SpecFlow.TestRunnerManager.OnTestRunnerEnd()
   at TechTalk.SpecFlow.TestRunnerManager.<InitializeBindingRegistry>b__19_0(Object <p0>, EventArgs <p1>)
========== Run test finished: 1 run (0:00:41.9274236) ==========

Next Attempt-
Updating SpecFlow to 2.3.0 did not help to resolve this error. Any other suggestions?

[naeemakram]

I installed an older version, v2017.2.1 to be precise and things started to flow perfectly again.

[mhossen]

I have .net Core NUnit project on Visual Studio 2017. I am able to run my Selenium Test Project without any issue when running from VS 2017 solution. But I am having same issue when I try to run on TeamCity build trigger. I was able to run test using vstest configuration for .Net but now its broken again.
dotnet vstest (19s) [15:30:24] .NET SDK 2.2.105 C:\Program Files\dotnet\dotnet.exe vstest DotNetCoreTeamCityIntergrationTests\DotNetCoreTeamCityIntergrationTests\bin\Debug\netcoreapp2.2\DotNetCoreTeamCityIntergrationTests.dll /logger:logger://teamcity /TestAdapterPath:C:\TeamCity\buildAgent\plugins\dotnet\tools\vstest15 /logger:console;verbosity=diagnostic [15:30:24] Starting: C:\Program Files\dotnet\dotnet.exe vstest DotNetCoreTeamCityIntergrationTests\DotNetCoreTeamCityIntergrationTests\bin\Debug\netcoreapp2.2\DotNetCoreTeamCityIntergrationTests.dll /logger:logger://teamcity /TestAdapterPath:C:\TeamCity\buildAgent\plugins\dotnet\tools\vstest15 /logger:console;verbosity=diagnostic [15:30:24] in directory: C:\TeamCity\buildAgent\work\a27db0e4325b6dd2 [15:30:24] Microsoft (R) Test Execution Command Line Tool Version 15.9.0 [15:30:24] Copyright (c) Microsoft Corporation. All rights reserved. [15:30:24] [15:30:24] Starting test execution, please wait... [15:30:27] Starting ChromeDriver 83.0.4103.39 (ccbf011cb2d2b19b506d844400483861342c20cd-refs/branch-heads/4103@{#416}) on port 60857 [15:30:27] Only local connections are allowed. [15:30:27] Please see https://chromedriver.chromium.org/security-considerations for suggestions on keeping ChromeDriver safe. [15:30:27] ChromeDriver was started successfully. [15:30:43] DotNetCoreTeamCityIntergrationTests.dll [15:30:43] DotNetCoreTeamCityIntergrationTests.Features.SiteContentFeature.UserIsAbleToViewSiteContentPage [15:30:43] System.NullReferenceException : Object reference not set to an instance of an object. [15:30:43] at DotNetCoreTeamCityIntergrationTests.Hooks.WebDriverHook.BrowserSetup() in C:\TeamCity\buildAgent\work\a27db0e4325b6dd2\DotNetCoreTeamCityIntergrationTests\DotNetCoreTeamCityIntergrationTests\Hooks\WebDriverHook.cs:line 27 at TechTalk.SpecFlow.Bindings.BindingInvoker.InvokeBinding(IBinding binding, IContextManager contextManager, Object[] arguments, ITestTracer testTracer, TimeSpan& duration) in D:\a\1\s\TechTalk.SpecFlow\Bindings\BindingInvoker.cs:line 69 at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.InvokeHook(IBindingInvoker invoker, IHookBinding hookBinding, HookType hookType) in D:\a\1\s\TechTalk.SpecFlow\Infrastructure\TestExecutionEngine.cs:line 352 at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.FireEvents(HookType hookType) in D:\a\1\s\TechTalk.SpecFlow\Infrastructure\TestExecutionEngine.cs:line 338 at DotNetCoreTeamCityIntergrationTests.Features.SiteContentFeature.ScenarioStart() at DotNetCoreTeamCityIntergrationTests.Features.SiteContentFeature.UserIsAbleToViewSiteContentPage() in C:\TeamCity\buildAgent\work\a27db0e4325b6dd2\DotNetCoreTeamCityIntergrationTests\DotNetCoreTeamCityIntergrationTests\Features\SiteContent.feature:line 4 [15:30:43] Failed UserIsAbleToViewSiteContentPage [15:30:43] Error Message: [15:30:43] System.NullReferenceException : Object reference not set to an instance of an object. [15:30:43] Stack Trace: [15:30:43] at DotNetCoreTeamCityIntergrationTests.Hooks.WebDriverHook.BrowserSetup() in C:\TeamCity\buildAgent\work\a27db0e4325b6dd2\DotNetCoreTeamCityIntergrationTests\DotNetCoreTeamCityIntergrationTests\Hooks\WebDriverHook.cs:line 27 [15:30:43] at TechTalk.SpecFlow.Bindings.BindingInvoker.InvokeBinding(IBinding binding, IContextManager contextManager, Object[] arguments, ITestTracer testTracer, TimeSpan& duration) in D:\a\1\s\TechTalk.SpecFlow\Bindings\BindingInvoker.cs:line 69 [15:30:43] at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.InvokeHook(IBindingInvoker invoker, IHookBinding hookBinding, HookType hookType) in D:\a\1\s\TechTalk.SpecFlow\Infrastructure\TestExecutionEngine.cs:line 352 [15:30:43] at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.FireEvents(HookType hookType) in D:\a\1\s\TechTalk.SpecFlow\Infrastructure\TestExecutionEngine.cs:line 338 [15:30:43] at DotNetCoreTeamCityIntergrationTests.Features.SiteContentFeature.ScenarioStart() [15:30:43] at DotNetCoreTeamCityIntergrationTests.Features.SiteContentFeature.UserIsAbleToViewSiteContentPage() in C:\TeamCity\buildAgent\work\a27db0e4325b6dd2\DotNetCoreTeamCityIntergrationTests\DotNetCoreTeamCityIntergrationTests\Features\SiteContent.feature:line 4 [15:30:43] Test Run Failed. [15:30:43] Total tests: 1. Passed: 0. Failed: 1. Skipped: 0. [15:30:43] Test execution time: 18.3031 Seconds [15:30:43] Process exited with code 1 [15:30:43] Process finished with positive exit code 1 (some tests have failed). Reporting step success as all the tests have run.

[SabotageAndi]

@mhossen Please create a new issue. Your comment has nothing to do with this issue. This issue is about the Visual Studio Integration.

[SabotageAndi]

Closed because of inactivity

[github-actions]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.