Running tests tries to run .NET Framework tests with mono on non-Windows #81395

vcsjones · 2023-01-30T23:40:22Z

When I run tests on my Mac now, it seems to be trying to run .NET Framework tests with mono. For example, running this command on my Mac

./dotnet.sh test src/libraries/System.Security.Cryptography.Pkcs/tests

It will first run all of the .NET (Core) tests and everything passes. Great. Then it proceeds to run the .NET Framework test, and gives this error:

A total of 1 test files matched the specified pattern.
System.IO.FileNotFoundException: Could not find 'mono' host. Make sure that 'mono' is installed on the machine and is available in PATH environment variable.
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Helpers.DotnetHostHelper.GetMonoPath()
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DefaultTestHostManager.GetTestHostProcessStartInfo(IEnumerable`1 sources, IDictionary`2 environmentVariables, TestRunnerConnectionInfo connectionInfo)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, String runSettings)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyExecutionManager.StartTestRun(TestRunCriteria testRunCriteria, IInternalTestRunEventsHandler eventHandler)

Even if I install mono, the tests will absolutely crash and burn since most of the project I work on that target .NET Framework assume they are running on Windows. Running S.S.C.Pkcs on macOS with Mono with the .NET Framework target has hundreds of failures.

Was there a change that introduced this recently, and, was it intentional? Or did something change in my environment that's making it try to do this?

The text was updated successfully, but these errors were encountered:

dotnet-issue-labeler · 2023-01-30T23:40:25Z

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

ghost · 2023-01-30T23:40:45Z

Tagging subscribers to this area: @dotnet/runtime-infrastructure
See info in area-owners.md if you want to be subscribed.

Issue Details

When I run tests on my Mac now, it seems to be trying to run .NET Framework tests with mono. For example, running this command on my Mac

./dotnet.sh test src/libraries/System.Security.Cryptography.Pkcs/tests

It will first run all of the .NET (Core) tests and everything passes. Great. Then it proceeds to run the .NET Framework test, and gives this error:

A total of 1 test files matched the specified pattern.
System.IO.FileNotFoundException: Could not find 'mono' host. Make sure that 'mono' is installed on the machine and is available in PATH environment variable.
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Helpers.DotnetHostHelper.GetMonoPath()
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DefaultTestHostManager.GetTestHostProcessStartInfo(IEnumerable`1 sources, IDictionary`2 environmentVariables, TestRunnerConnectionInfo connectionInfo)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, String runSettings)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyExecutionManager.StartTestRun(TestRunCriteria testRunCriteria, IInternalTestRunEventsHandler eventHandler)

Even if I install mono, the tests will absolutely crash and burn since most of the project I work on that target .NET Framework assume they are running on Windows. Running S.S.C.Pkcs on macOS with Mono with the .NET Framework target has hundreds of failures.

Was there a change that introduced this recently, and, was it intentional? Or did something change in my environment that's making it try to do this?

Author:	vcsjones
Assignees:	-
Labels:	`question`, `area-Infrastructure`, `untriaged`
Milestone:	-

ViktorHofer · 2023-01-31T15:58:47Z

Probably regressed with dotnet/arcade@e82404f. I will take a look in the next days.

ghost · 2023-01-31T15:59:13Z

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

Issue Details

When I run tests on my Mac now, it seems to be trying to run .NET Framework tests with mono. For example, running this command on my Mac

./dotnet.sh test src/libraries/System.Security.Cryptography.Pkcs/tests

It will first run all of the .NET (Core) tests and everything passes. Great. Then it proceeds to run the .NET Framework test, and gives this error:

A total of 1 test files matched the specified pattern.
System.IO.FileNotFoundException: Could not find 'mono' host. Make sure that 'mono' is installed on the machine and is available in PATH environment variable.
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Helpers.DotnetHostHelper.GetMonoPath()
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Hosting.DefaultTestHostManager.GetTestHostProcessStartInfo(IEnumerable`1 sources, IDictionary`2 environmentVariables, TestRunnerConnectionInfo connectionInfo)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, String runSettings)
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyExecutionManager.StartTestRun(TestRunCriteria testRunCriteria, IInternalTestRunEventsHandler eventHandler)

Even if I install mono, the tests will absolutely crash and burn since most of the project I work on that target .NET Framework assume they are running on Windows. Running S.S.C.Pkcs on macOS with Mono with the .NET Framework target has hundreds of failures.

Was there a change that introduced this recently, and, was it intentional? Or did something change in my environment that's making it try to do this?

Author:	vcsjones
Assignees:	ViktorHofer
Labels:	`question`, `area-Infrastructure-libraries`
Milestone:	8.0.0

Bartleby2718 · 2023-05-14T00:58:42Z

@ViktorHofer Do you happen to have any updates on this?

I think my real question is: what's the suggested way to exclude .NET Framework tests? AFAIK exactly one argument is expected for the --framework flag of dotnet test, so I'm not sure if there's a project-agnostic way of excluding .NET Framework tests in a multi-targeting project.

vcsjones · 2023-05-14T03:21:21Z

I'm not sure if there's a project-agnostic way of excluding .NET Framework tests in a multi-targeting project.

I've been doing something like this for now.

./dotnet.sh test src/libraries/Microsoft.Bcl.Cryptography/tests /p:TargetFramework=net8.0

I can't exclude .NET Framework, but I can run a particular framework, which works for my purposes.

ViktorHofer · 2023-05-15T15:44:57Z

I had to put this on the back-burner, unfortunately. We could either change this directly only for runtime or move this issue over into the vstest repository and ask for a switch to not invoke Mono when testing .NET Framework. These days, running .NET Framework via Mono on Unix platforms is a niche scenario.

ViktorHofer · 2023-05-15T15:45:10Z

cc @MarcoRossignoli @nohwnd

MarcoRossignoli · 2023-05-15T16:10:43Z

Looks like is there since 2017 and for what I recall we didn't to any update to this part of the resolution alg(netfx)

@nohwnd I think you weren't here at that time too but do you have some information about? Have you seen issue like that in past? Do you think it's safe to remove?
I'd say yet because since I'm here I didn't see issue related to Mono(something on coverage but I'd say unrelated) but I'd like to have your thoughts before.

https://github.com/microsoft/vstest/pull/851/files#diff-b989eb0b8261eb128f64dbe8d9139b357b53310858f40655f657934a5dd19b42R152

ViktorHofer · 2023-05-15T17:24:47Z

Just to clarify, I wasn't asking for your team to make a breaking change and remove VSTest's .NET Framework Mono support (which could still be done, but might require more discussion). I was asking if it would be possible to expose a flag when invoking dotnet test (which would also be set via a runsettings variable and/or msbuild property) to disable that code path.

nohwnd · 2023-05-30T10:35:40Z

It would be great if this could be handled on the msbuild level, ideally we should run all the dlls that you give us to run, and in this case we get both .net and .net framework dlls because the build that happens under dotnet test (when used with csproj or sln) is building all the multi-targets.

Implementing this on vstest level would mean adding a tfm filter, which I don't think is impossible, but might be a bit confusing to use, and because we historically succeed when stuff is not run, we should have do it here as well and it would make ignoring your whole test-suite quite easy.

E.g. naming the allowed frameworks --framework-filter: dotnet test --framework net8.0 --framework-filter net6.0, net7.0

This would filter your dlls only to net6.0. and net7.0 dlls from run that targets only net8.0, so tests from 0 assemblies are run. We can warn about --framework and --framework-filter not having any intersection, and fail, but then it is not consistent with many of the other parameters that allow you to get a warning and continue.

We'd try to use Mono to run them but most of the tests assume Windows when targetting net4x so they don't work anyway and you'd need to remember to pass the `-f net9.0` TFM filter. Instead just skip them on non-Windows platforms. Fixes dotnet#81395

* Don't run net4x tests with Mono on non-Windows We'd try to use Mono to run them but most of the tests assume Windows when targetting net4x so they don't work anyway and you'd need to remember to pass the `-f net9.0` TFM filter. Instead just skip them on non-Windows platforms. Fixes #81395 * Make AdditionalProperties more resilient against changes * Use TargetOS instead of OS so it also works for building e.g. Browser targets on Windows * Move logic to arcade Microsoft.DotNet.Build.Tasks * Update eng/testing/outerBuild.targets Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com> --------- Co-authored-by: Viktor Hofer <viktor.hofer@microsoft.com>

Bartleby2718 · 2024-03-02T12:40:55Z

@akoeplinger @ViktorHofer Thanks for fixing this! Given that .NET 8 is an LTS, is there any chance this can be backported to .NET 8?

akoeplinger · 2024-03-02T13:13:36Z

@Bartleby2718 #98767 only affects development inside of the dotnet/runtime repository so it's not likely we'll backport it.

ghost added the untriaged New issue has not been triaged by the area owner label Jan 30, 2023

vcsjones added question Answer questions and provide assistance, not an issue with source code or documentation. area-Infrastructure labels Jan 30, 2023

ViktorHofer self-assigned this Jan 31, 2023

ViktorHofer removed the untriaged New issue has not been triaged by the area owner label Jan 31, 2023

ViktorHofer added this to the 8.0.0 milestone Jan 31, 2023

ViktorHofer added area-Infrastructure-libraries and removed area-Infrastructure question Answer questions and provide assistance, not an issue with source code or documentation. labels Jan 31, 2023

ViktorHofer modified the milestones: 8.0.0, Future Jul 20, 2023

ViktorHofer removed their assignment Jul 20, 2023

akoeplinger self-assigned this Feb 21, 2024

akoeplinger mentioned this issue Feb 21, 2024

Don't run net4x tests with Mono on non-Windows #98767

Merged

ghost added the in-pr There is an active PR which will close this issue when it is merged label Feb 21, 2024

akoeplinger mentioned this issue Feb 26, 2024

Make running netfx assemblies via Mono opt-in on non-Windows platforms microsoft/vstest#4908

Closed

ViktorHofer closed this as completed in #98767 Mar 2, 2024

ghost removed the in-pr There is an active PR which will close this issue when it is merged label Mar 2, 2024

github-actions bot locked and limited conversation to collaborators Apr 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Running tests tries to run .NET Framework tests with mono on non-Windows #81395

Running tests tries to run .NET Framework tests with mono on non-Windows #81395

vcsjones commented Jan 30, 2023

dotnet-issue-labeler bot commented Jan 30, 2023

ghost commented Jan 30, 2023

ViktorHofer commented Jan 31, 2023

ghost commented Jan 31, 2023

Bartleby2718 commented May 14, 2023

vcsjones commented May 14, 2023

ViktorHofer commented May 15, 2023

ViktorHofer commented May 15, 2023

MarcoRossignoli commented May 15, 2023 •

edited

Loading

ViktorHofer commented May 15, 2023 •

edited

Loading

nohwnd commented May 30, 2023

Bartleby2718 commented Mar 2, 2024 •

edited

Loading

akoeplinger commented Mar 2, 2024 •

edited

Loading

Running tests tries to run .NET Framework tests with mono on non-Windows #81395

Running tests tries to run .NET Framework tests with mono on non-Windows #81395

Comments

vcsjones commented Jan 30, 2023

dotnet-issue-labeler bot commented Jan 30, 2023

ghost commented Jan 30, 2023

ViktorHofer commented Jan 31, 2023

ghost commented Jan 31, 2023

Bartleby2718 commented May 14, 2023

vcsjones commented May 14, 2023

ViktorHofer commented May 15, 2023

ViktorHofer commented May 15, 2023

MarcoRossignoli commented May 15, 2023 • edited Loading

ViktorHofer commented May 15, 2023 • edited Loading

nohwnd commented May 30, 2023

Bartleby2718 commented Mar 2, 2024 • edited Loading

akoeplinger commented Mar 2, 2024 • edited Loading

MarcoRossignoli commented May 15, 2023 •

edited

Loading

ViktorHofer commented May 15, 2023 •

edited

Loading

Bartleby2718 commented Mar 2, 2024 •

edited

Loading

akoeplinger commented Mar 2, 2024 •

edited

Loading