2.1.300-preview2: Unable to use DotNetCliToolReference due to deps.json generation failure

[natemcmaster]

There appears to be a regression in DotNetCliToolReference support.

Repro
On a clean mac, install the .NET Core CLI using the .pkg installer (not .tar.gz)

Create a new project via dotnet new xunit

Run dotnet xunit

Expected

Should launch xunit

Actual

The SDK fails to launch it with this error.

Unable to generate deps.json, it may have been already generated. You can specify the "-d" option before the tool name for diagnostic output (for example, "dotnet -d ": /usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/GenerateDeps/GenerateDeps.proj

The real issue seems to be this:

error MSB4018: System.UnauthorizedAccessException: Access to the path '/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/GenerateDeps/obj/Debug/netcoreapp2.1/GenerateDeps.assets.cache' is denied.

The .pkg installer does not give write access to this path.

ls -al /usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/GenerateDeps
total 4
drwxr-xr-x  4 root wheel  128 Apr  6 23:18 .
drwxr-xr-x 36 root wheel 1152 Apr  6 23:18 ..
-rwxr-xr-x  1 root wheel 2950 Apr  6 10:11 GenerateDeps.proj
drwxr-xr-x  3 root wheel   96 Apr  6 23:18 obj

Workaround

sudo chmod -R a+w /usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/GenerateDeps

Details
Using 2.1.300-preview2-008530

Telemetry is: Enabled
projectfactory: MSBUILD_EXE_PATH = /usr/local/share/dotnet/sdk/2.1.300-preview2-008530/MSBuild.dll
projectfactory: MSBuild project path = /private/tmp/tst/tst.csproj
projecttoolscommandresolver: resolving commandspec from 1 Tool Libraries.
projecttoolscommandresolver: Attempting to resolve command spec from tool dotnet-xunit
projecttoolscommandresolver: nuget packages root:
- /Users/nmcmaster/.nuget/packages/
- /usr/local/share/dotnet/sdk/NuGetFallbackFolder
projecttoolscommandresolver: found tool lockfile at : /Users/nmcmaster/.nuget/packages/.tools/dotnet-xunit/2.3.1/netcoreapp2.1/project.assets.json
projecttoolscommandresolver: expect deps.json at: /Users/nmcmaster/.nuget/packages/.tools/dotnet-xunit/2.3.1/netcoreapp2.1/dotnet-xunit.deps.json
Generating deps.json at: /Users/nmcmaster/.nuget/packages/.tools/dotnet-xunit/2.3.1/netcoreapp2.1/dotnet-xunit.deps.json
MSBuild arguments: /usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/GenerateDeps/GenerateDeps.proj /p:ProjectAssetsFile=\"/Users/nmcmaster/.nuget/packages/.tools/dotnet-xunit/2.3.1/netcoreapp2.1/project.assets.json\" /p:ToolName=dotnet-xunit /p:ProjectDepsFilePath=/var/folders/wt/qgc8g0h10y356ch360468p4r0000gn/T/tmpqVwU2r.tmp /p:TargetFramework=netcoreapp2.1
Unable to generate deps.json, it may have been already generated.  You can specify the "-d" option before the tool name for diagnostic output (for example, "dotnet -d <toolname>": Microsoft (R) Build Engine version 15.7.145.53551 for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.
/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Microsoft.Common.CurrentVersion.targets(1128,5): warning MSB3191: Unable to create directory "bin/Debug/netcoreapp2.1/". Access to the path '/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/GenerateDeps/bin/Debug/netcoreapp2.1/' is denied. [/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/GenerateDeps/GenerateDeps.proj]
/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018: The "ResolvePackageAssets" task failed unexpectedly. [/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/GenerateDeps/GenerateDeps.proj]
/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018: System.UnauthorizedAccessException: Access to the path '/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/GenerateDeps/obj/Debug/netcoreapp2.1/GenerateDeps.assets.cache' is denied. ---> System.IO.IOException: Permission denied [/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/GenerateDeps/GenerateDeps.proj]
/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018:    --- End of inner exception stack trace --- [/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/GenerateDeps/GenerateDeps.proj]
/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018:    at Interop.ThrowExceptionForIoErrno(ErrorInfo errorInfo, String path, Boolean isDirectory, Func`2 errorRewriter) [/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/GenerateDeps/GenerateDeps.proj]
/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018:    at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String path, OpenFlags flags, Int32 mode) [/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/GenerateDeps/GenerateDeps.proj]
/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018:    at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share, Int32 bufferSize, FileOptions options) [/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/GenerateDeps/GenerateDeps.proj]
/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018:    at System.IO.FileStream..ctor(String path, FileMode mode, FileAccess access, FileShare share) [/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/GenerateDeps/GenerateDeps.proj]
/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018:    at System.IO.File.Open(String path, FileMode mode, FileAccess access, FileShare share) [/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/GenerateDeps/GenerateDeps.proj]
/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018:    at Microsoft.NET.Build.Tasks.ResolvePackageAssets.CacheWriter..ctor(ResolvePackageAssets task) [/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/GenerateDeps/GenerateDeps.proj]
/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018:    at Microsoft.NET.Build.Tasks.ResolvePackageAssets.CacheReader..ctor(ResolvePackageAssets task) [/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/GenerateDeps/GenerateDeps.proj]
/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018:    at Microsoft.NET.Build.Tasks.ResolvePackageAssets.ReadItemGroups() [/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/GenerateDeps/GenerateDeps.proj]
/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018:    at Microsoft.NET.Build.Tasks.ResolvePackageAssets.ExecuteCore() [/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/GenerateDeps/GenerateDeps.proj]
/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018:    at Microsoft.NET.Build.Tasks.TaskBase.Execute() [/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/GenerateDeps/GenerateDeps.proj]
/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/GenerateDeps/GenerateDeps.proj]
/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/Microsoft.PackageDependencyResolution.targets(205,5): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [/usr/local/share/dotnet/sdk/2.1.300-preview2-008530/Sdks/Microsoft.NET.Sdk/targets/GenerateDeps/GenerateDeps.proj]

[livarcocc]

@dsplaisted please take a look ASAP.

[dsplaisted]

This looks like for this tool scenario, ResolvePackageAssets (which is new) is trying to write the assets cache to a location it doesn’t have write access to. Tests missed this because the SDK folder isn’t write protected when the tests run.

[dsplaisted]

@nguerrera @livarcocc Possible solutions:

• Add a mode to ResolvePackageAssets where it doesn't write out a cache to disk (it could write and then read it to a memory stream to preserve the same logic flow as today)
• Set the BaseIntermediateOutputPath to a writeable path when building GenerateDeps.proj. This could be a temp path or possibly a path under the BaseIntermediateOutputPath of the project the tool is referenced from

Thoughts?

[nguerrera]

This explains some test flakiness too, I think.

[nguerrera]

I think opting in to a no-write-mode makes sense. @dsplaisted I can grab this if you're not already working on it.

[nguerrera]

Fixed by #2134

[shanselman]

• Does this have any performance ramifications for builds?
• Does this mean that xunit testing is effectively broken in Preview2 (I hit this immediately) and are we set up to find this kind of thing if it breaks again?

[nguerrera]

There should be no performance impact to build. This changes a one time action to generate tool deps json but inner loop builds will still utilize the same package assets cache as before.

There's an update to preview 2 going out with this fixed.

There is unit test coverage added to prevent regression.