"Run SpecFlow Scenarios" does not work with ReSharper 9 #435

gasparnagy · 2015-04-30T08:23:31Z

"Run SpecFlow Scenarios" does not work with ReSharper 9.

See https://groups.google.com/forum/#!topic/specflow/fmxx8nZPxx8 for details

gasparnagy · 2015-04-30T08:27:07Z

I've updated the commands (see commit SpecFlowOSS/SpecFlow.VisualStudio@e250890), but it did not help...

It shows an error like

[10:26:08.1571693] ReSharper6TestRunnerGateway: test tool error: System.Runtime.InteropServices.COMException (0x80004005): Command "ReSharper.Reharper_ReSharperUnitTestRunContext" is not valid.
   at EnvDTE._DTE.ExecuteCommand(String CommandName, String CommandArgs)
   at TechTalk.SpecFlow.VsIntegration.TestRunner.CommandBasedTestRunnerGateway.RunInCurrentContext(Boolean debug)

I have no further idea. I fear that this will not work (it was a hack anyway)...

If anyone has a good idea, go ahead and try to fix it.

davidmcgregor · 2015-05-07T02:48:51Z

I believe your commit has a typo -
"ReSharper.Reharper_ReSharperUnitTestRunContext"
should be
"ReSharper.ReSharper_ReSharperUnitTestRunContext"

Hope it helps.

lindavz28 · 2015-05-10T23:31:12Z

Not sure if this is related or not, but this command doesn't work with Resharper 8.2 either. It's not Resharper 9 specific. I had a working system, upgraded to latest "Specflow for Visual Studio 2013" v2015.1.1 and started getting the "No tests found in file" error.

Specflow trace:
[11:37:29.1871281] ReSharper6TestRunnerGateway: test tool error: System.Runtime.InteropServices.COMException (0x80004005): Command "ReSharper.Reharper_ReSharperUnitTestDebugContext" is not valid.
at EnvDTE._DTE.ExecuteCommand(String CommandName, String CommandArgs)
at TechTalk.SpecFlow.VsIntegration.TestRunner.CommandBasedTestRunnerGateway.RunInCurrentContext(Boolean debug)
[11:37:29.3071401] ReSharper6TestRunnerGateway: test tool error: System.Runtime.InteropServices.COMException (0x80004005): Command "ReSharper.Reharper_ReSharperUnitTestDebugContext" is not valid.
at EnvDTE._DTE.ExecuteCommand(String CommandName, String CommandArgs)
at TechTalk.SpecFlow.VsIntegration.TestRunner.CommandBasedTestRunnerGateway.RunInCurrentContext(Boolean debug)

lindavz28 · 2015-05-11T04:14:42Z

OK, starting to understand the problem here.
Resharper6+ supports this command:

Resharper9 has changed to ReSharper.Resharper_ReSharperUnitTestDebugContext

From what I can see, you can't just change the Resharper6 class - that will break all current builds using Resharper 8.2 - which is what I've seen today. It's trying to run ReSharper.Reharper_ReSharperUnitTestDebugContext

(Edited) Better idea - you can look up the version of Resharper being used, and adjust the command accordingly.

I'm thinking something like this...
I have made the change and will do a PR if I can test it properly (First time working with VS extensions)

public int GetResharperVersion()
{
            var reSharperAssembly = AppDomain.CurrentDomain.GetAssemblies().FirstOrDefault(a => a.GetName().Name == "JetBrains.ReSharper.UnitTestFramework");
            return reSharperAssembly != null ? reSharperAssembly.GetName().Version.Major : -1;
}

protected override string GetRunInCurrentContextCommand(bool debug)
{
            // This should be stored in constructor so we don't run it all the time
            int currentVersion = GetResharperVersion();
            string commandFormat;
            if (currentVersion < 6)
            {
                commandFormat = "ReSharper.ReSharper_UnitTest_Context{0}";
            }
            else if (currentVersion < 9)
            {
                commandFormat = "ReSharper.ReSharper_ReSharper_UnitTest_{0}Context";
            }
            else
            {
                commandFormat = "ReSharper.ReSharper_ReSharperUnitTest{0}Context";
            }

            if (debug)
            {
                return string.Format(commandFormat, "Debug");
            }
            return string.Format(commandFormat, "Run");

}

gasparnagy · 2015-05-11T08:22:59Z

@lindavz28 yes. please try to do it.

lindavz28 · 2015-05-11T10:49:05Z

@gasparnagy SpecFlowOSS/SpecFlow.VisualStudio#4
Only tested VS2013 + Resharper 8.2 to check that's working again.

(Edit) - Figured it out, need to set experimental hive option in RS9 installer. Will test tomorrow morning

gasparnagy · 2015-05-11T14:38:24Z

@lindavz28 super, thx

lindavz28 · 2015-05-11T22:43:04Z

@gasparnagy Made my life a bit harder by trying to get XUnit working on RS9, but successfully ran specflow tests with RS9! 👍 😄

gasparnagy · 2015-05-12T08:02:11Z

@lindavz28 super! thx. I merged it.
Let's wait for a week, if there is any issue pops up with 2015.1 and if not, I'll update the VS gallery.
I close this issue now.

lindavz28 · 2015-05-22T00:09:30Z

@gasparnagy When will you be updating the VS gallery?

gasparnagy · 2015-05-26T05:57:34Z

I have updated the VS gallery with the new version now.

stickleprojects · 2015-05-27T09:58:49Z

HI
I've uninstalled and updated to the v2015.1.2 and experiencing the same issue in vs2013/r#9.
Is there further uninstall steps I need to follow?

stickleprojects · 2015-05-27T10:26:10Z

Hi
Same problem, vs2010/r#9 (platform v6.0.2):

[11:23:47.6127725] ReSharper6TestRunnerGateway: test tool error: System.Runtime.InteropServices.COMException (0x80004005): Command "ReSharper.ReSharper_ReSharper_UnitTest_DebugContext" is not valid.
at EnvDTE._DTE.ExecuteCommand(String CommandName, String CommandArgs)
at TechTalk.SpecFlow.Vs2010Integration.TestRunner.CommandBasedTestRunnerGateway.RunInCurrentContext(Boolean debug)

resharper debug information, in case it is helpful:

JetBrains ReSharper Platform 6 Build 6.0.20141219.120158 on 2014-12-19 12:56:02Z
JetBrains ReSharper 9.0 Update 1 Build 9.0.20141219.130142 on 2014-12-19 13:12:31Z
JetBrains TeamCity Add-in 9.0 Update 1 Build 9.0.20141219.130134 on 2014-12-19 13:13:21Z

Application Packages:
JetBrains Fake VisualStudio VsPackageAssembly build 10.0.0.0 on 2015-03-19 13:38:37Z.
JetBrains Platform Core Ide build 6.0.20141219.120158 on 2014-12-19 12:56:02Z.
JetBrains Platform Core Shell build 6.0.20141219.120162 on 2014-12-19 12:53:09Z.
JetBrains Platform Core Text build 6.0.20141219.120158 on 2014-12-19 12:56:02Z.
JetBrains Platform Installer build 6.0.20141219.120158 on 2014-12-19 12:56:02Z.
JetBrains Platform Symbols build 6.0.20141219.120158 on 2014-12-19 12:56:02Z.
JetBrains Platform VisualStudio build 6.0.20141219.120158 on 2014-12-19 12:56:02Z.
JetBrains Psi.Features ExternalAnnotations build 9.0.20141219.130140 on 2014-12-19 13:00:40Z.
JetBrains Psi.Features SolutionBuilder build 9.0.20141219.130140 on 2014-12-19 13:00:40Z.
JetBrains Psi.Features src build 9.0.20141219.130140 on 2014-12-19 13:00:40Z.
JetBrains Psi.Features Tasks build 6.0.20141219.130140 on 2014-12-19 13:00:40Z.
JetBrains Psi.Features TeamCity src build 9.0.20141219.130134 on 2014-12-19 13:13:21Z.
JetBrains Psi.Features VisualStudio build 9.0.20141219.130140 on 2014-12-19 13:00:40Z.
JetBrains ReSharper src build 9.0.20141219.130142 on 2014-12-19 13:12:31Z.
Visual Studio 10.0.30319.1.

samholder · 2015-05-27T12:30:57Z

hit reply on the wrong email @shaundodimead? 😆

shaundodimead · 2015-05-27T12:57:36Z

Yup :) fml - there's a github feature I didn't know existed...

lindavz28 · 2015-05-29T23:34:01Z

The plugin being used is:TechTalk.SpecFlow.Vs2010Integration

My fixes were in the newer SpecFlow.VisualStudio plugin.

I'm guessing this is not backwards compatible with 2010/2012... I'll need
to do the same fix on this as well then....

On Wed, May 27, 2015 at 10:26 PM, stickleprojects notifications@github.com
wrote:

Hi
Same problem, vs2010/r#9 (platform v6.0.2):

[11:23:47.6127725] ReSharper6TestRunnerGateway: test tool error:
System.Runtime.InteropServices.COMException (0x80004005): Command
"ReSharper.ReSharper_ReSharper_UnitTest_DebugContext" is not valid.
at EnvDTE._DTE.ExecuteCommand(String CommandName, String CommandArgs)
at
TechTalk.SpecFlow.Vs2010Integration.TestRunner.CommandBasedTestRunnerGateway.RunInCurrentContext(Boolean
debug)

resharper debug information, in case it is helpful:

JetBrains ReSharper Platform 6 Build 6.0.20141219.120158 on 2014-12-19
12:56:02Z
JetBrains ReSharper 9.0 Update 1 Build 9.0.20141219.130142 on 2014-12-19
13:12:31Z
JetBrains TeamCity Add-in 9.0 Update 1 Build 9.0.20141219.130134 on
2014-12-19 13:13:21Z

Application Packages:
JetBrains Fake VisualStudio VsPackageAssembly build 10.0.0.0 on 2015-03-19
13:38:37Z.
JetBrains Platform Core Ide build 6.0.20141219.120158 on 2014-12-19
12:56:02Z.
JetBrains Platform Core Shell build 6.0.20141219.120162 on 2014-12-19
12:53:09Z.
JetBrains Platform Core Text build 6.0.20141219.120158 on 2014-12-19
12:56:02Z.
JetBrains Platform Installer build 6.0.20141219.120158 on 2014-12-19
12:56:02Z.
JetBrains Platform Symbols build 6.0.20141219.120158 on 2014-12-19
12:56:02Z.
JetBrains Platform VisualStudio build 6.0.20141219.120158 on 2014-12-19
12:56:02Z.
JetBrains Psi.Features ExternalAnnotations build 9.0.20141219.130140 on
2014-12-19 13:00:40Z.
JetBrains Psi.Features SolutionBuilder build 9.0.20141219.130140 on
2014-12-19 13:00:40Z.
JetBrains Psi.Features src build 9.0.20141219.130140 on 2014-12-19
13:00:40Z.
JetBrains Psi.Features Tasks build 6.0.20141219.130140 on 2014-12-19
13:00:40Z.
JetBrains Psi.Features TeamCity src build 9.0.20141219.130134 on
2014-12-19 13:13:21Z.
JetBrains Psi.Features VisualStudio build 9.0.20141219.130140 on
2014-12-19 13:00:40Z.
JetBrains ReSharper src build 9.0.20141219.130142 on 2014-12-19 13:12:31Z.
Visual Studio 10.0.30319.1.

Copyright © 2015 JetBrains. All rights reserved.

—
Reply to this email directly or view it on GitHub
#435 (comment).

gasparnagy · 2015-05-30T06:49:10Z

@lindavz28 yes, the update is made for VS2013 and VS2015. but @stickleprojects mentioned also VS2013. Now I'm confused.
Is it working now in VS2013 latest released version on VS gallery or not? And if not: was the fix not sufficient or I did something wrong with the release?

(I can make an updated release until Wed, but then I'm off for two weeks.)

stickleprojects · 2015-05-30T07:19:03Z

Further investigation, I noticed that Resharper did not have Keyboard commands similar to "unit", "test", "debug" mentioned above. Reinstall of latest version of R#9 fixed the issue in VS2013. Tested using latest Github source and Gallery install.

lpanger · 2015-06-05T17:00:03Z

FYI, it works for me.

specflow plugin 2015.1.2
resharper version 9.1 (not 9.1.1)
visual studio 12.0.31101.00 update 4

MattiLehtinen · 2015-09-07T10:31:30Z

I get slightly different error:

ReSharper6TestRunnerGateway: test tool error: System.Runtime.InteropServices.COMException (0x80004005): Command "ReSharper.ReSharper_UnitTestRunContext" is not available.
at EnvDTE._DTE.ExecuteCommand(String CommandName, String CommandArgs)
at TechTalk.SpecFlow.VsIntegration.TestRunner.CommandBasedTestRunnerGateway.RunInCurrentContext(Boolean debug)

SpecFlow for Visual Studio 2015 2015.1.2
VS 2015 14.0.23107.0
ReSharper 9.2

Tried to reinstall specflow and resharper with no success. Any tips?

lindavz28 · 2015-09-08T09:24:51Z

@MattiLehtinen Go to Tools > Options > Keyboard bindings (Might be under Editor section).
Search for Resharper and then UnitTestRunContext (or something like that)
Have a look at what the command is showing - is it ReSharper.ReSharper_UnitTestRunContext or something else? What are you seeing instead?

Unless Resharper has gone and changed the command again for Resharper 9.2, it should work....

Final thing you can try is uninstall Resharper and install the specific version 9.1.3:
http://resharper-support.jetbrains.com/hc/en-us/articles/206104048-Where-can-I-download-an-old-previous-ReSharper-version-

MattiLehtinen · 2015-09-08T10:20:30Z

@lindavz28 The command seems to be there:

I get the same error with VS2013 too.

MattiLehtinen · 2015-09-14T12:15:05Z

@lindavz28 I installed Resharper 9.1.3 to VS2013 and it works. So I guess there's some breaking change on ReSharper 9.2.

lindavz28 · 2015-09-15T08:36:25Z

@MattiLehtinen Glad you managed to get it working! I suspect a breaking change in 9.2 as well, though it's interesting that the command looks identical to what's in the options list. It might also be a bug on the Specflow side... I'll have a look at 9.2 when I get a chance.

dbgrvorgau · 2015-09-24T22:47:21Z

Hi any news on getting resharper 9.2 running with visual studio 2010 and specflow?

Gonnagle · 2015-10-20T06:56:43Z

Had the same issue as @MattiLehtinen with VS 2013, ReSharper 9.2 and SpecFlow 2015.1.2. Fixed for now by reverting back to ReSharper 9.1.3.

sergiorykov · 2015-11-11T09:20:40Z

The same issue in VS 2015, ReSharper (9.2 and 10) and SpecFlow 2015.1.2

Ctrl-T,R/D works from *.feature-file as a pretty useful workaround, but I occasionally find myself right-clicking and searching menu item :)

rubig · 2015-12-08T05:40:29Z

Hi guys,

Visual Studio version does not matter. What matters is the Resharper version.

Resharper 9.2+ not working with Specflow 2015. The reason is:

Specflow uses Resharper 9.1.2's key for running unit test:
ReSharper.ReSharper_ReSharper_UnitTest_RunContext

While on ReSharper 9.2+ the key has changed into
ReSharper.ReSharper_UnitTest_RunContext

I'll submit a pull request to get Specflow updated with Resharper 9.2+.

The code change will be something like:

Note 2 difference my code vs lindavz28's code

ReSharper version that divides how unit test run command works is 9.2, not 9.

ReSharper 9.2+ command for running unit test is ReSharper.ReSharper_UnitTest{0}Context, not ReSharper.ReSharper_ReSharperUnitTest{0}Context

protected override string GetRunInCurrentContextCommand(bool debug)
{
    // This should be stored in constructor so we don't run it all the time
    int currentVersion = GetResharperVersion();
    string commandFormat;
    if (currentVersion < 6)
    {
        commandFormat = "ReSharper.ReSharper_UnitTest_Context{0}";
    }
    else if (currentVersion < 9.2)
    {
        commandFormat = "ReSharper.ReSharper_ReSharper_UnitTest_{0}Context";
    }
    else
    {
        commandFormat = "ReSharper.ReSharper_UnitTest{0}Context";
    }

    if (debug)
    {
        return string.Format(commandFormat, "Debug");
    }
    return string.Format(commandFormat, "Run");

}

Cheers,
Ming

lindavz28 · 2015-12-21T04:06:15Z

How did that update go @rubig, did you test your update using Resharper 9.1 and 9.2?

I don't follow your logic though - my code as follows - any version that is 9 or more should use the command you've mentioned. Doesn't matter if it's 9.1, 9.2 or 9.3

protected override string GetRunInCurrentContextCommand(bool debug)
        {
            string commandFormat;
            if (_currentVersion < 6)
            {
                commandFormat = "ReSharper.ReSharper_UnitTest_Context{0}";
            }
            else if (_currentVersion < 9)
            {
                commandFormat = "ReSharper.ReSharper_ReSharper_UnitTest_{0}Context";
            }
            else
            {
                commandFormat = "ReSharper.ReSharper_UnitTest{0}Context";
            }

            if (debug)
            {
                return string.Format(commandFormat, "Debug");
            }
            return string.Format(commandFormat, "Run");

        }

Resharper 9 changed to use the following:
ReSharper.ReSharper_UnitTestDebugContext
ReSharper.ReSharper_UnitTestRunContext

That should be the case for 9.1, 9.2 and 9.3. There's no difference between 9.1 and 9.2 - if there was I would've implemented a fix immediately. That's the problem :)

Comparing 9.2:

With 9.1.3:

So I have 9.1.3 installed in my main VS 2015 (not the hive) and the command is identical.
Have you been able to debug the extension and actually follow through the logic to see what command is being used?

lindavz28 · 2015-12-21T04:08:43Z

@rubig I would love to get this fixed btw, so if you have got something working please send that PR through 👍

Latest discussion here - please continue in this thread rather than the closed issue
#477

I have the stack trace and the command being run is "ReSharper.ReSharper_UnitTestRunContext" It works if you run it in a command window, but not from the Specflow extension. 😭

gasparnagy · 2015-12-21T07:04:19Z

@rubig @lindavz28 yes please. don't forget that the PR should be sent to the https://github.com/techtalk/specflow.visualstudio repo.

dorothyvaliga · 2016-01-26T19:01:15Z

Upvoting this please 👍 -- it degrades the user experience for SpecFlow! 😭

gasparnagy · 2016-01-26T19:21:17Z

We release SpecFlow v2 tomorrow and the next step will be to make an update on the VS integration.

lazytesting · 2016-01-27T14:35:56Z

I agree with @dorothyvaliga. I recently upgraded tot VS2015 en ReSharper 10 and although there are workarounds available it's very frustrating that it's not possible to run the features from the context menu.

dorothyvaliga · 2016-01-29T14:34:47Z

Working now with VS 15 Specflow 2 Resharper 9.1.3

darrencauthon · 2016-01-29T15:24:18Z

Thank you @dorothyvaliga ! :)

How about you @lazytesting ?

lazytesting · 2016-01-29T15:42:50Z

It's now working partially for me with V2 (in combination with NUnit 3):

When I open the feature file and right click in the editor window and select run/debug still nothing happens (no errors either).

When I run them from the solution explorer (with right click on the feature file) the tests are working.
But when I don't have the feature file opened in VS I also get an error popup "Object reference not set to an instance of an object." but the tests are still running.

lindavz28 · 2016-01-29T23:03:46Z

@darrencauthon @dorothyvaliga @lazytesting
Has been working for a while with 9.1.3 in V1. Something changed in version 9.2 of Resharper to make it break, but I thought it was only xunit affected. Looks like it's NUnit as well which means a deeper issue than just the command being run by Specflow.

See this issue which is the most current thread:
#477

@gavinlees said:
@lindavz28 The issue showed up in when Jetbrains fixed this bug in R# 9.3 EAP build #3:
https://youtrack.jetbrains.com/issue/RSRP-446021

rubig · 2016-01-31T05:06:10Z

Nice! I’ll give it a try on Monday.

rubig · 2016-02-01T05:33:02Z

@lindavz28
I am sorry for my late reply. I dived into Specflow source code, and I found Specflow 1.9.3 already contains the same fix I was going to add.

However, I can confirm this version of Specflow still has issue in running tests with Resharper 9.2.

This leads me to think my solution above is not the correct fix for this issue. I'll give Specflow 2 a try and see if it can run nicely with Resharper 9.2.

crmckenzie · 2016-03-15T18:22:45Z

I'm using Specflow 2 with Resharper 10 in VS 2015. Any word on how to make this work?

IcodeNet · 2016-04-20T12:45:07Z

Hi all - I am using
JetBrains ReSharper Ultimate 10.0.2 Build 104.0.20151218.120627
ReSharper 10.0.20151218.130009

and SpecFLow 2015.1.2 - 2015/05/26

Any ideas about how to make this work for my pair? Thanks in advance

SabotageAndi · 2016-04-20T20:19:07Z

@IcodeNet Keyboard shortcuts could work.
Have a look also at #477

IcodeNet · 2016-04-27T10:45:25Z

Hi guys I have managed to resolve it in my case by selecting in Specflow options menu Resharper as a test runner tool. It was Auto to start off .

hope it works for you as well

lazytesting · 2016-04-27T12:09:20Z

@IcodeNet for me that didn't work. But it helped me finding a better workaround. I set the Test Runner Tool to VisualStudio2012 and installed the Nunit VS plugin. So I'm now able to run the test from the context menus. But as they don't run in ReSharper I don't have the ReSharper test result view.

folke123 · 2016-09-29T18:41:38Z

Does not work for Reshaper 2016.2.2

Pressing "run unit tests" in the context menu gives the error message "The operation could not be completed. Not implemented"

The same error if you do it by keyboard shortcut.

I am running VS2015 and the "Specflow for Visual Studio 2015 plugin"
and we are using Nunit 2.6.3 as a nuget for the solution

SamJongenelen · 2017-03-07T15:39:09Z

Same here, does not work. Set Test Runner Tool to Resharper doesn't work.
Setting to VS 2012 works.

lock · 2019-07-09T09:29:36Z

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

gasparnagy added Bug out-of-scope and removed out-of-scope labels Apr 30, 2015

gasparnagy closed this as completed May 12, 2015

lock bot locked as resolved and limited conversation to collaborators Jul 9, 2019

"Run SpecFlow Scenarios" does not work with ReSharper 9 #435

"Run SpecFlow Scenarios" does not work with ReSharper 9 #435

Comments

gasparnagy commented Apr 30, 2015

gasparnagy commented Apr 30, 2015

davidmcgregor commented May 7, 2015

lindavz28 commented May 10, 2015

lindavz28 commented May 11, 2015

gasparnagy commented May 11, 2015

lindavz28 commented May 11, 2015

gasparnagy commented May 11, 2015

lindavz28 commented May 11, 2015

gasparnagy commented May 12, 2015

lindavz28 commented May 22, 2015

gasparnagy commented May 26, 2015

stickleprojects commented May 27, 2015

stickleprojects commented May 27, 2015

samholder commented May 27, 2015

shaundodimead commented May 27, 2015

lindavz28 commented May 29, 2015

gasparnagy commented May 30, 2015

stickleprojects commented May 30, 2015

lpanger commented Jun 5, 2015

MattiLehtinen commented Sep 7, 2015

lindavz28 commented Sep 8, 2015

MattiLehtinen commented Sep 8, 2015

MattiLehtinen commented Sep 14, 2015

lindavz28 commented Sep 15, 2015

dbgrvorgau commented Sep 24, 2015

Gonnagle commented Oct 20, 2015

sergiorykov commented Nov 11, 2015

rubig commented Dec 8, 2015

lindavz28 commented Dec 21, 2015

lindavz28 commented Dec 21, 2015

gasparnagy commented Dec 21, 2015

dorothyvaliga commented Jan 26, 2016

gasparnagy commented Jan 26, 2016

lazytesting commented Jan 27, 2016

dorothyvaliga commented Jan 29, 2016

darrencauthon commented Jan 29, 2016

lazytesting commented Jan 29, 2016

lindavz28 commented Jan 29, 2016

rubig commented Jan 31, 2016

rubig commented Feb 1, 2016

crmckenzie commented Mar 15, 2016

IcodeNet commented Apr 20, 2016

SabotageAndi commented Apr 20, 2016

IcodeNet commented Apr 27, 2016

lazytesting commented Apr 27, 2016

folke123 commented Sep 29, 2016

SamJongenelen commented Mar 7, 2017

lock bot commented Jul 9, 2019