Build in visual studio fails due to incomplete package restore when projects are unloaded

[AArnott]

When a project is unloaded in Visual Studio, it ceases to build with solution build. But so long as it was built previously (either at the command line or while loaded in Visual Studio) the rest of the solution should build successfully even across project references that include this unloaded project because the binaries are on disk.

But with NuGet and project.json, this has regressed. When a project is unloaded that is in the middle of a chain of project references and that chain is what propagates dependencies from a project.json file at a root project, that causes solution build to fail. When the solution build starts, nuget restore rewrites all the project.json files an does so differently than a command line "nuget restore" command. In VS, it seems package restore is based on loaded VS projects, so the unloaded VS project causes a partial package restore failure. In particular, the project.json dependencies do not properly propagate to the projects that reference the unloaded project, resulting in build failures in these projects due to missing assembly references.

Here is sample output from one such failed solution build (while CodeGeneration.Roslyn.Tests is unloaded):

Restoring NuGet packages...
To prevent NuGet from restoring packages during build, open the Visual Studio Options dialog, click on the Package Manager node and uncheck 'Allow NuGet to download missing packages during build.'
Failed to resolve all project references for 'CodeGeneration.Roslyn.Tests.Generators'. The package restore result for 'CodeGeneration.Roslyn.Tests.Generators' may be incomplete.
Failed to resolve all project references for 'CodeGeneration.Roslyn.Tasks'. The package restore result for 'CodeGeneration.Roslyn.Tasks' may be incomplete.
1>------ Build started: Project: CodeGeneration.Roslyn.Tasks, Configuration: Debug Any CPU ------
2>------ Build started: Project: CodeGeneration.Roslyn.NuGet, Configuration: Debug Any CPU ------
1>C:\Users\andarno\git\CodeGeneration.Roslyn\src\CodeGeneration.Roslyn.Tasks\GenerateCodeFromAttributes.cs(17,34,17,41): error CS0234: The type or namespace name 'MSBuild' does not exist in the namespace 'Microsoft.CodeAnalysis' (are you missing an assembly reference?)
1>C:\Users\andarno\git\CodeGeneration.Roslyn\src\CodeGeneration.Roslyn.Tasks\GenerateCodeFromAttributes.cs(18,34,18,38): error CS0234: The type or namespace name 'Text' does not exist in the namespace 'Microsoft.CodeAnalysis' (are you missing an assembly reference?)
1>C:\Users\andarno\git\CodeGeneration.Roslyn\src\CodeGeneration.Roslyn.Tasks\GenerateCodeFromAttributes.cs(177,21,177,28): error CS0246: The type or namespace name 'Project' could not be found (are you missing a using directive or an assembly reference?)
========== Build: 1 succeeded, 1 failed, 2 up-to-date, 0 skipped ==========

Note this repros even when I've successfully run a "nuget restore" and build at the command line, then immediately try to build in VS. So very obviously VS is rewriting the project.lock.json files when building, resulting in this failure.

This breaks scenarios the benefit or even require unloaded projects (such as in my case, where a loaded project results in assemblies getting loaded and locked in the devenv.exe process which breaks the build which rewrites that assembly).

Repro steps

1. Create a new solution.
2. Add class library A project and a project.json to it.
3. Add a dependency on the nuget Validation package to that project.json file (or any package that delivers assemblies).
4. Add class library B, and a project.json to it.
5. Add a project reference from B to A.
6. Add class library C, and a project.json to it.
7. Add a project reference from C to B.
8. Add a class to class library C and a method that calls Validation.Assumes.True(true);
9. Build the solution. Everything works.
10. Unload project B.
11. Build the solution. This time, the build breaks because Validation.Assumes cannot be found.
12. At the command line, observe that "nuget restore" followed by a command line build of the solution works.
13. Build the solution in VS again. The build is still broken.

[yishaigalatzer]

This is somewhat expected behavior of the current codebase, it walks the projects through DTE, and that just doesn't work when the projects are unloaded. We could look at improving this, however a simple workaround is available.

Turn off automatic package restore (through nuget settings) and nuget restore doesn't kick off again and "break the chain".

The assumption is that unloaded project don't necessary compile or work, and hence we didn't try to follow down their path.

If we do attempt to fix this, is there a mechanism to get P2P through DTE for unloaded projects?

[AArnott]

VS offers no services for unloaded projects other than to discover their project factory GUID and filename. But perhaps you could use that information to read project data from disk or whatever you would do from the command line for that project? If not, what data do you get from DTE that you can't get in another way?

[yishaigalatzer]

Command line runs a full msbuild evaluation for the solution. It is really slow and I don't think it is a good idea to run it in this mode at least not at the current performance level.

[AArnott]

True, in fact the command line mode does much more than evaluation. It actually executes MSBuild targets, which is likely where you lose most of your perf. A single evaluation of an MSBuild project is on the order of 20ms, so that shouldn't tax you too much IMO, even if you did it for all projects in VS. But doing it for just the unloaded projects would have a very small perf hit IMO for the accuracy it affords.

[nkolev92]

I believe this is has been addressed by NuGet/NuGet.Client#2521, which was work for the solution filters feature.

The original issue linked to that PR is #5820, but this one is equally as appropriate.

If a restore has run successfully (either through a commandline restore or a restore where all projects are loaded in VS), NuGet will reuse the old "knowledge" about the transitive projects and do a best effort restore.

Created an issue to increase the visiblity of this feature as it's come up many times in issues/discussions: https://github.com/NuGet/docs.microsoft.com-nuget/issues/2031