Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentation about known issue regarding xunit tests when targeting .NET Full Framework #371

Closed
AdmiringWorm opened this issue May 8, 2019 · 7 comments
Milestone

Comments

@AdmiringWorm
Copy link
Member

When trying to run unit test on Linux, and you have either is using multiple target frameworks, or is just targeting a Full .NET Framework, then the .NET Core test utility is unable to actually find/run the unit tests (It will still run unit tests for .NET Core and .NET Standard).

I know this is a known problem in the .NET Core repository, and the reason why this happen can be different between different linux distros (in some it may actually succeed as well). In my case, it is because the testhost adapter that .NET Core tries to run, don't exist anywhere (and no way of getting a hold of it).

As such, I suggest that when running on a Unix platform, run only .NET Core targeted frameworks using dotnet test. But run the correct unit test executable for frameworks targeting the full .NET Framework (like what is done when running a normal MSBuild).

@gep13 gep13 added this to the 2.0.0 milestone Sep 9, 2019
@gep13
Copy link
Member

gep13 commented Jul 7, 2020

@AdmiringWorm is this still an issue?

@AdmiringWorm
Copy link
Member Author

I think it is, but haven't really tested it since I created the issue.
In my projects I am disabling full framework tests on Unix platforms.

@AdmiringWorm
Copy link
Member Author

Just added .NET Framework to the .NET Core unit tests, and it seems to be still an issue.

At the very least xUnit fails for .NET Framework, and it looks like there is something wrong for NUnit tests as well (although no failure here).
https://travis-ci.org/github/AdmiringWorm/Cake.Recipe.Tests/jobs/707742236#L3089

@gep13
Copy link
Member

gep13 commented Jul 14, 2020

Doh! 😢

@AdmiringWorm
Copy link
Member Author

AdmiringWorm commented Jul 14, 2020

On a positive note though, MSTest unit tests seems to work out-of-the-box 😄

@gep13
Copy link
Member

gep13 commented Jul 14, 2020

Woot! 🎉

@AdmiringWorm
Copy link
Member Author

By doing some different testing regarding this issue, I found that there isn't much we can/should do in cake.recipe itself. But I believe this is an issue we would need/want to document (maybe in a known issues section?).

Now onto what the issue actually is:

The issue seems to only be related to the xunit testing framework when targeting a .NET Full Framework version and running on Unix (not confirmed if it is just Linux, or if macosx is also affected, but I believe so).
For the package, there is an implicit dependency on Microsoft.TestPlatform.ObjectModel which gets pulled in automatically when running on windows, but unfortunately do not on Unix.
Because of this, this dependency is needed to be explicitly referenced in the unit testing project.
The best way I found to do this, was to only restore the dependency when running .NET Full Framework and on Unix with the following csproj addition.

<PackageReference Include="Microsoft.TestPlatform.ObjectModel" Version="16.6.1" Condition="$(TargetFramework.StartsWith('net4')) AND '$(OS)' == 'Unix'" />

The above package reference version should also match the package reference version of Microsoft.NET.Test.Sdk

The error text that a user gets without this package reference may look similar to:

[xUnit.net 00:00:00.76] XUnitTests: Catastrophic failure: System.TypeLoadException: Could not load type of field 'Xunit.Runner.VisualStudio.VsExecutionSink:recorder' (4) due to: Could not load file or assembly 'Microsoft.VisualStudio.TestPlatform.ObjectModel, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies.

@AdmiringWorm AdmiringWorm changed the title dotnet test is unable to run unit tests for full .NET Frameworks Add documentation about known issue regarding xunit tests when targeting .NET Full Framework Jul 14, 2020
gep13 added a commit that referenced this issue Aug 26, 2020
cake-contrib-bot pushed a commit that referenced this issue Aug 26, 2020
(GH-371) Added known issues section
@gep13 gep13 closed this as completed Aug 27, 2020
cake-contrib-bot pushed a commit that referenced this issue Aug 27, 2020
(GH-371) Add info about known Xunit issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants