Regression in 5.0.20x SDK when building with --no-dependencies

[yaakov-h]

When building a project with --no-dependencies that references another project, when both projects build to the same output directory, the SDK deletes the .deps.json, .runtimeconfig.dev.json, and .runtimeconfig.json files during the build process and then produces errors because those files could not be copied (as they do not exist).

Repro repository: https://github.com/yaakov-h/dotnetsdk5.20x-error

To reproduce this, clone the above repository and then run .\build.ps1. Output for me is:

PS C:\Data\Temp\sdk20x-error> .\build.ps1
  Determining projects to restore...
  All projects are up-to-date for restore.
Microsoft (R) Build Engine version 16.9.0+57a23d249 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  My.Cool.Website -> C:\Data\Temp\sdk20x-error\bin\net5.0\My.Cool.Website.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:01.13
  Determining projects to restore...
  All projects are up-to-date for restore.
Microsoft (R) Build Engine version 16.9.0+57a23d249 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

C:\Program Files\dotnet\sdk\5.0.200\Microsoft.Common.CurrentVersion.targets(4919,5): error MSB3030: Could not copy the file "C:\Data\Temp\sdk20x-error\bin\net5.0\My.Cool.Website.deps.json" because it was not found. [C:\Data\Temp\sdk20x-error\My.Cool.Website.Tests\My.Cool.Website.Tests.csproj]
C:\Program Files\dotnet\sdk\5.0.200\Microsoft.Common.CurrentVersion.targets(4919,5): error MSB3030: Could not copy the file "C:\Data\Temp\sdk20x-error\bin\net5.0\My.Cool.Website.runtimeconfig.dev.json" because it was not found. [C:\Data\Temp\sdk20x-error\My.Cool.Website.Tests\My.Cool.Website.Tests.csproj]
C:\Program Files\dotnet\sdk\5.0.200\Microsoft.Common.CurrentVersion.targets(4919,5): error MSB3030: Could not copy the file "C:\Data\Temp\sdk20x-error\bin\net5.0\My.Cool.Website.runtimeconfig.json" because it was not found. [C:\Data\Temp\sdk20x-error\My.Cool.Website.Tests\My.Cool.Website.Tests.csproj]

Build FAILED.

C:\Program Files\dotnet\sdk\5.0.200\Microsoft.Common.CurrentVersion.targets(4919,5): error MSB3030: Could not copy the file "C:\Data\Temp\sdk20x-error\bin\net5.0\My.Cool.Website.deps.json" because it was not found. [C:\Data\Temp\sdk20x-error\My.Cool.Website.Tests\My.Cool.Website.Tests.csproj]
C:\Program Files\dotnet\sdk\5.0.200\Microsoft.Common.CurrentVersion.targets(4919,5): error MSB3030: Could not copy the file "C:\Data\Temp\sdk20x-error\bin\net5.0\My.Cool.Website.runtimeconfig.dev.json" because it was not found. [C:\Data\Temp\sdk20x-error\My.Cool.Website.Tests\My.Cool.Website.Tests.csproj]
C:\Program Files\dotnet\sdk\5.0.200\Microsoft.Common.CurrentVersion.targets(4919,5): error MSB3030: Could not copy the file "C:\Data\Temp\sdk20x-error\bin\net5.0\My.Cool.Website.runtimeconfig.json" because it was not found. [C:\Data\Temp\sdk20x-error\My.Cool.Website.Tests\My.Cool.Website.Tests.csproj]
    0 Warning(s)
    3 Error(s)

Time Elapsed 00:00:01.06

(The additional --no-* arguments in the build script are not necessary to trigger the failure, but I included them in the test case because that is how we build our projects.)

This project builds successfully with the .NET 5.0.103 and 5.0.104 SDKs, but fails on the .NET 5.0.200 and 5.0.201 SDKs.

I suspect this was caused by #14488.

My suspicion is that additional json files for My.Cool.Website are now being considered as output of My.Cool.Website.Tests, so it includes them in the Clean build target towards the very beginning of the build.

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[marcpopMSFT]

I think your analysis of what's going on is accurate. We don't think this rises in priority to the level of us fixing this for the upcoming releases.

[yaakov-h]

Which upcoming releases are you referring to? A 5.0.2xx/5.0.3xx release, or .NET 6?

[yaakov-h]

I've tried working on a fix but ultimately I'm stuck.

I'd like to start by saying I absolutely love the test framework here and it made writing the test surprisingly easy. Good job!

Finding a fix for the problem was more tricky. The test showed that initial analysis above was not entirely correct:

1. The code seems to build fine the first time but fail the second time. Although this doesn't quite match what I was seeing in production, it does match my repro case - from a clean repo (once I cleared out the accidentally-committed obj folder) my .\build.ps1 script passes the first time and fails the second time. I may have to dig a bit deeper here to see why the repro fails on the second attempt but in my production build environment it failed on the first attempt.
2. --no-incremental is a key part of the issue, it was not solely a problem with --no-dependencies. A viable workaround is to remove --no-incremental.

My attempt so far can be found here: main...yaakov-h:yaakov/fix16299

---

The approach that I want to take is quite simple: If the SDK is asked to copy a file from our own output directory we skip copying it. This is because:

1. It was clearly not part of our build process, since it got there before our build actually built anything. It is therefore not our responsibility to clean etc.
2. If it is already in the output directory then the copy operation becomes a no-op anyways. The file is already exactly where it needs to be. There is no actual work for us to do.

Unfortunately, while I can skip copying it, I can't figure out how to stop cleaning it. Therefore, even though my dependent project now builds, the necessary files still get deleted and the referenced project's output won't run. Unless I can skip deleting the files too, this approach won't get anywhere.

---

Another approach, which would possibly be more correct:

I have noticed that the SDK is smart enough to not delete the .dll/.pdb/etc. files from the referenced project, but it still deletes the .runtimeconfig.json/.runtimeconfig.dev.json/.deps.json files.

It looks like that decision is made somewhere else, not in this repo (possibly in MSBuild repo?) so I can't see how to extend that check. If we can include these files in the set of files related to the assembly that we already know not to delete, then the problem should similarly disappear.

Any suggestions?