[Bug]: /warnaserror false positive overall success

[AArnott]

Issue Description

I'm not sure if this is an msbuild issue or an issue with GitHub Actions OSX agents. The problem is unique to OSX though.

As you can see from the following GitHub Actions run, all 3 jobs experienced the same build errors (which were warnings-as-errors, BTW), but only Windows and Ubuntu reported failure. The Mac OSX agent went on and reported overall success, defeating the point of my setting MSBuildTreatWarningsAsErrors to true as an environment variable.

https://github.com/AArnott/Nerdbank.Cryptocurrencies/actions/runs/4905363640/jobs/8759094126?pr=5

Steps to Reproduce

git clone https://github.com/AArnott/Nerdbank.Cryptocurrencies
cd Nerdbank.Cryptocurrencies
git checkout 52b604bdd0ed2239ca491118ee7a886ab60963d1

Then push this commit to some branch and run github actions on your fork and observe the results.

Expected Behavior

I expect all jobs to fail.

Actual Behavior

The OSX job reports success, despite its own build log reporting several errors.

Analysis

No response

Versions & Configurations

No response

[JaynieBai]

CentralNode_MSBuild_PID=22676_x64_BuildManager_Default.log

This should be msbuild issue. Repro this locally both on window and mac. haven't try it on Linux. But not sure why this is only repro on GitHub Actions OSX.
Repro steps locally:

git clone https://github.com/AArnott/Nerdbank.Cryptocurrencies
cd Nerdbank.Cryptocurrencies
git checkout 52b604bdd0ed2239ca491118ee7a886ab60963d1
msbuild /p:MSBuildTreatWarningsAsErrors=true /t:rebuild

when build with msbuild /p:MSBuildTreatWarningsAsErrors=true /t:rebuild /m:1, It build failed as expected all the time. But with multiple CPU counts greater than 2, the issues will repro.

[rainersigwald]

It looks like specifying -warnaserror on the command line works correctly on macOS. That's generally preferable to setting the property via environment variable for CI scenarios (since it works for all categories of warning and doesn't hit #4038).

However, what you're doing should work and it's very confusing that it is wrong only on macOS. We should drill into that.

[AR-May]

Some information we found on the issue:

The issue reproduces not only on MacOS, I managed to reproduce it on Windows as well. It seems to be a race condition in build manager from long time ago, @rokonec soon will make a fix for it. Just recently the probability of it was increased and we started to see it more. Note, if you use .NET Core version of msbuild ("dotnet build" command instead of "msbuild"), the bug often goes away. This might work as a temporary workaround.

Easy repro steps:

1. Create a new console application.
2. Add a class library as a second project to the solution.
3. Make class library to throw a warning (add unused variable for example)
4. Build with msbuild /p:MSBuildTreatWarningsAsErrors=true /t:rebuild /m:2