Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nunit 2 test adapter does not support Visual Studio 2019 #180

Closed
bkilada opened this issue Apr 5, 2019 · 16 comments

Comments

@bkilada
Copy link

commented Apr 5, 2019

The NUnit 2 test adapter doesn't support Visual Studio 2019, which was released a few days ago. Are there any plans implement support?

Thanks,
Ben

@OsirisTerje

This comment has been minimized.

Copy link
Member

commented Apr 5, 2019

It is supported by the nuget adapter, but not the vsix. The number of vsix downloads are very low now, so , yes, we can update it, it doesn't take much, but is it really needed ?

As mentioned other places, we would really like to see people moving to the nuget adapter, and support for vsix will be deprecated in the VS anyway going forward.

@n3wjack

This comment has been minimized.

Copy link

commented Apr 12, 2019

For some reason the nuget adapter isn't always enough. It stopped working for me in 2018 at some point.
Running all tests is OK, but if I want to run a specific test, it doesn't or run them. It can't discover them for some reason according to the output window.
So I resorted to the vsix, which fixed that issue in 2018.
In 2019 I now have the same issue, but no vsix to fix that with. :(

@bkilada

This comment has been minimized.

Copy link
Author

commented Apr 14, 2019

Ah, I didn't know there is a NuGet adapter. I tried that but unfortunately ran into similar issues to @n3wjack. I run a test and it immediately fails with no error message. The NUnit 3 NuGet adapter had the same problem, only the NUnit 3 extension worked.

@n3wjack

This comment has been minimized.

Copy link

commented Apr 14, 2019

It's a weird problem. In the VS test explorer the tests stay blue as in "not run tests". So the tests are discovered, but when you select one or more, or use the shortcut keys to run the tests in the current open file, the test run starts but doesn't find any tests.
The funny thing is that when you use "run all tests" everything works.

@OsirisTerje

This comment has been minimized.

Copy link
Member

commented Apr 14, 2019

Can you upload a small repro?

@n3wjack

This comment has been minimized.

Copy link

commented Apr 15, 2019

This behaves the same. It works in VS2017, with the vsix test adapter installed.
In VS2019, it doesn't run the tests, but it does discover them.

I installed the nuget packages in this order:

  • nunit 2
  • nunit testadapter

nunit2_testadapter_vs2019.zip

@OsirisTerje

This comment has been minimized.

Copy link
Member

commented Apr 15, 2019

@n3wjack

  1. The vsix doesn't work with VS 2019. We'll try to get out a new release with that supported shortly.
  2. You have included the nuget adapter, but the vsix takes precendence.
    If you uninstall the vsix, then the nuget adapter should work.
    Can you check if all works then?
    (3. The discovery you see is probably not from the adapter, but from the source based real time discovery in the test explorer. )
@n3wjack

This comment has been minimized.

Copy link

commented Apr 15, 2019

I know the VSIX doesn't work in 2019, it's not in the plugin list, which is why I ended up here. :)

I now disabled the VSIX in VS2017 and now I have the same behavior there as in VS2019.
Tests are discovered (you see them in the test explorer), but aren't run when I want to run an individual test or use "Run all".
This is exactly why I installed the VSIX in 2017.

@OsirisTerje

This comment has been minimized.

Copy link
Member

commented Apr 15, 2019

@n3wjack Thanks for the repo and the explanations!
I can confirm the bug, a bit surprised about it in fact.
I'll come back to you on this as soon as possible.
Thanks again, really appreciate this !

@n3wjack

This comment has been minimized.

Copy link

commented Apr 15, 2019

@OsirisTerje

This comment has been minimized.

Copy link
Member

commented Apr 15, 2019

UPDATE:
Visual Studio should manage to discover the adapters when they are in the packages folder, but it seems there is some change there. You mention it stopped working "some time" in 2018, and I think that was a minor update change.

In the options for Test it says:

image

(2) above explains why you get it working with the vsix, which currently doesn't help you with VS2019.
(1) above explains why it don't find it, because the 2.1 adapter doesn't copy itself to the binary output of the test project, which it should do.

I am pretty sure that before some time in 2018, VS managed to find them under the packages folder, but that doesn't help much now.

WORKAROUND:
What you can do, is to add a runsettings file:
image
where you set the adapter path to some local folder where you have manually copied the adapter dll's:
image

When I do that with your repo, it starts working again:
image

This is only a workaround, but should work both locally and on a build server.

We need to fix this in the adapter too.
The following can be done:

  1. Update the vsix to support installation in VS2019. I think we should do this.
  2. Add the copying of the nuget adapter dlls to the test project outputs the same way as it is done in the NUnit3 adapter. I think we should do this too.

@OsirisTerje OsirisTerje added pri:high and removed easyfix labels Apr 15, 2019

@OsirisTerje OsirisTerje self-assigned this Apr 15, 2019

@OsirisTerje OsirisTerje removed the pri:low label Apr 15, 2019

@OsirisTerje OsirisTerje added this to the 2.2 milestone Jun 2, 2019

OsirisTerje pushed a commit that referenced this issue Jun 2, 2019

@OsirisTerje OsirisTerje closed this Jun 3, 2019

@OsirisTerje

This comment has been minimized.

Copy link
Member

commented Jun 5, 2019

@n3wjack @bkilada The pre-release package for the 2.2 adapter is now available on https://www.myget.org/feed/nunit/package/nuget/NUnitTestAdapter If you could check it out, it would be awesome.

@bkilada

This comment has been minimized.

Copy link
Author

commented Jun 6, 2019

@OsirisTerje

This comment has been minimized.

Copy link
Member

commented Jun 6, 2019

@bkilada It is now out on nuget, as version 2.2.0, so it is released :-) If you find anything, give a ping. :-)

@n3wjack

This comment has been minimized.

Copy link

commented Jun 6, 2019

Just upgraded the package in our solution and the unittests are now discovered and run in VS2019. 👍
Thanks for fixing this!

@OsirisTerje

This comment has been minimized.

Copy link
Member

commented Jun 6, 2019

@n3wjack Cool! Thanks !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.