Support VS test runner

[adamralph]

I'm not sure what others are using for running tests in VS. Historically I've always used TestDriven.NET which works just fine. Recently, on my new ultrabook, I've decided to give the VS test runner another spin.

There are several issues currently:

• FakeItEasy.IntegrationTests.RaisingEventWithMultipleSubscribers fails when running all tests, but passes when run in isolation
• The IsValid_should_return_false_for_invalid_values test which FakeItEasy.Tests.ExpressionsConstraints.EqualityArgumentConstraintTests inherits from FakeItEasy.Tests.ArgumentConstraintTestBase is not run with the message Source: no source available
• The Windows Store 8 test introduced in Support for Windows Store test projects #214 is not run with the message Unit tests in source C:\code\FakeItEasy\Source\FakeItEasy.Win8.Tests\bin\Debug\FakeItEasy.Win8.Tests.dll cannot be run. Please create a Windows Store or Windows Phone Unit Test project.

Even though I'm happy to continue using TestDriven.NET, I think we should encourage contribution by supporting the VS test runner completely.

[PureKrome]

Adam, not sure if this could help or not .. but ... you're using NUnit and i THOUGHT that when you run any 2+ tests in the same run, they use the same AppDomain. So, for example, you stick something into a static .. then the next test will re-use that static value from the previous test.

In other words: tests are not run in true Isolation.

Now, xUnit was sorta suppose to fix this .. but i've found this problem recently when doing some stuff with Logging (which uses static crap) -and- a modern computer where ... i can have MULTIPLE tests running at the same time, during the same run (aka session). This was achieved using xUnit + Resharper .. where resharper has an option to say: run x number of tests concurrently.

So it looked like even xUnit wasn't 100% right, also.

FWIW - just some info to help you debug this issue.

[adamralph]

Thanks @PureKrome. Not sure what to make of it all at the moment! 😕

[adamralph]

We should hold off on this one until #545 is closed.

[blairconrad]

Is this still valid? The VSIX extension isn't supported anymore. Do we need to use xunit.runner.visualstudio to run the tests via the VS test runner? If so, unless we want to add that package to FIE, is there much point?

In related news,

And all but ApproveApi pass in isolation.

[blairconrad]

I should've said. I was running the branch that has no NUnit, just xUnit tests.

[adamralph]

Yes, in order to support the VS runner after our migration to xunit, we'd have to install https://www.nuget.org/packages/xunit.runner.visualstudio in all our projects. I guess it depends on how much benefit this would convey to our contributors.

[blairconrad]

I'd asked about this in the xUnit slack and
@jedidja commented, suggesting xunit.extensibility.execution.

[adamralph]

suggesting xunit.extensibility.execution

I don't understand this. The solution for running xunit with the VS runner has been for some time now to install https://www.nuget.org/packages/xunit.runner.visualstudio, as detailed in the recently updated https://xunit.github.io/docs/getting-started-desktop.html#run-tests-visualstudio

When I raised the issue, I was trying to run the current NUnit tests using the VS runner. We may indeed find that after migrating to xunit and installing xunit.runner.visualstudio, everything just works.

xunit.extensibility.execution is for building extensions for xunit, such as xbehave.

[blairconrad]

I don't understand this

I don't understand anything.

Thanks for the link to the getting started. You may have just provided the solution to
xunit/xunit#887.

[blairconrad]

When I raised the issue, I was trying to run the current NUnit tests using the VS runner. We may indeed find that after migrating to xunit and installing xunit.runner.visualstudio, everything just works.

Perhaps. Especially now that we'll be disabling parallelization in what I think are the relevant assemblies.