Can't use `dotnet add package` with a private repository (basic authentication)

[a-h]

Details about Problem

• dotnet --version
  • 1.0.1
• dotnet
  • 1.1
• NuGet
  • 3.5.0.1996
• MacOS Sierra
  • 10.12.4

Steps to reproduce

1. Create a private NuGet repository (in my case using AppVeyor)
2. Add it as a source:

nuget sources add -Name xxx -Source https://ci.appveyor.com/nuget/xxx -Username xxx@xxxx.com -Password xxx

3. Prove that it's possible to list packages:

nuget list -s xxx

Using credentials from config. UserName: xxx@xxx.com
my.nuget.package 1.0.1

4 Attempt to add the private NuGet package to a dotnet project using the CLI:

dotnet add package -s https://ci.appveyor.com/nuget/xxx my.nuget.package

Expected

For the command to add the NuGet package to the project.

Actual

Got a message saying that it couldn't authenticate.

Microsoft (R) Build Engine version 15.1.548.43366
Copyright (C) Microsoft Corporation. All rights reserved.

  Writing /var/folders/v0/gv8rbbt9157g5599sh8qljpr0000gn/T/tmpz9doUg.tmp
info : Adding PackageReference for package 'my.nuget.package' into project '/Users/xxx/Documents/my.nuget.package/src/xxx/xxx.csproj'.
log  : Restoring packages for /Users/xxx/Documents/xxx/src/xxxLibrary/xxx.csproj...
info :   GET https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package'
info :   Unauthorized https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package' 4042ms
log  : Retrying 'FindPackagesByIdAsyncCore' for source 'https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package''.
log  : Response status code does not indicate success: 401 (Unauthorized).
info :   GET https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package'
info :   Unauthorized https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='xxx' 997ms
log  : Retrying 'FindPackagesByIdAsyncCore' for source 'https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='xxx'.
log  : Response status code does not indicate success: 401 (Unauthorized).
info :   GET https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package'
info :   Unauthorized https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='xxx' 1007ms
error: Failed to retrieve information about 'my.nuget.package' from remote source 'https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package''.
error:   Response status code does not indicate success: 401 (Unauthorized).

[a-h]

Really long winded workaround:

• mkdir download

• cd download

• nuget install my.nuget.package

• Wait for all the downloads to complete

• Note that it's been added to the cache:

  • ls /Users/xxx/.nuget/packages/my.nuget.package/

• Run dotnet restore from the solution directory, and it will work

[mishra14]

@a-h
I tried this scenario and I was able to install a package from a private feed using dotnet add -

F:\validation\test> dotnetlocal add package -s https://mishra14.pkgs.visualstudio.com/_packaging/temp/nuget/v3/index.json testapp
Microsoft (R) Build Engine version 15.2.93.5465
Copyright (C) Microsoft Corporation. All rights reserved.

  Writing C:\Users\anmishr\AppData\Local\Temp\tmp3568.tmp
info : Adding PackageReference for package 'testapp' into project 'F:\validation\test\test.csproj'.
log  : Restoring packages for F:\validation\test\test.csproj...
info :   GET https://mishra14.pkgs.visualstudio.com/_packaging/8e401993-92e2-4c09-8755-7c6003be8320/nuget/v3/flat2/testapp/index.json
info :   OK https://mishra14.pkgs.visualstudio.com/_packaging/8e401993-92e2-4c09-8755-7c6003be8320/nuget/v3/flat2/testapp/index.json 554ms
info :   GET https://mishra14.pkgs.visualstudio.com/_packaging/8e401993-92e2-4c09-8755-7c6003be8320/nuget/v3/flat2/testapp/1.0.0/testapp.1.0.0.nupkg
info :   OK https://mishra14.pkgs.visualstudio.com/_packaging/8e401993-92e2-4c09-8755-7c6003be8320/nuget/v3/flat2/testapp/1.0.0/testapp.1.0.0.nupkg 1213ms
log  : Installing TestApp 1.0.0.
info : Package 'testapp' is compatible with all the specified frameworks in project 'F:\validation\test\test.csproj'.
info : PackageReference for package 'testapp' version '1.0.0' added to file 'F:\validation\test\test.csproj'.


F:\validation\test> dotnetlocal --info
.NET Command Line Tools (2.0.0-preview2-005910)

Product Information:
 Version:            2.0.0-preview2-005910
 Commit SHA-1 hash:  f3a4b22457

Runtime Environment:
 OS Name:     Windows
 OS Version:  10.0.15063
 OS Platform: Windows
 RID:         win10-x64
 Base Path:   E:\cli\artifacts\win10-x64\stage2\sdk\2.0.0-preview2-005910

Microsoft .NET Core Shared Framework Host

  Version  : 2.0.0-preview2-002093-00
  Build    : 1a195046fd92bd73d993d804bea2f4506c2b56f1

The feed - https://mishra14.pkgs.visualstudio.com/_packaging/8e401993-92e2-4c09-8755-7c6003be8320/nuget/v3/flat2/testapp/index.json is a private feed.

Please make sure that the config file has the right credentials. In the mean time I will try with a V2 feed as well.

[mishra14]

I was able to add package from the V2 feed as well -

F:\validation\test> dotnetlocal add package -s https://mishra14.pkgs.visualstudio.com/_packaging/temp/nuget/v2 testapp
Microsoft (R) Build Engine version 15.2.93.5465
Copyright (C) Microsoft Corporation. All rights reserved.

  Writing C:\Users\anmishr\AppData\Local\Temp\tmp6E20.tmp
info : Adding PackageReference for package 'testapp' into project 'F:\validation\test\test.csproj'.
log  : Restoring packages for F:\validation\test\test.csproj...
info :   GET https://mishra14.pkgs.visualstudio.com/_packaging/temp/nuget/v2/FindPackagesById()?id='testapp'
info :   OK https://mishra14.pkgs.visualstudio.com/_packaging/temp/nuget/v2/FindPackagesById()?id='testapp' 1649ms
info :   GET https://mishra14.pkgs.visualstudio.com/_packaging/temp/nuget/v2?id=testapp&version=1.0.0
info :   OK https://mishra14.pkgs.visualstudio.com/_packaging/temp/nuget/v2?id=testapp&version=1.0.0 1084ms
log  : Installing TestApp 1.0.0.
info : Package 'testapp' is compatible with all the specified frameworks in project 'F:\validation\test\test.csproj'.
info : PackageReference for package 'testapp' version '1.0.0' updated in file 'F:\validation\test\test.csproj'.

[a-h]

OK, here's a script which reproduces the same problem inside a Linux container:

I created a file called test.sh.

mkdir TestProject
cd TestProject
echo "Creating new library..."
dotnet new library
echo "Installing Mono to run nuget.exe"
apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
echo "deb http://download.mono-project.com/repo/debian wheezy main" | tee /etc/apt/sources.list.d/mono-xamarin.list
echo "deb http://download.mono-project.com/repo/debian wheezy-apache24-compat main" | tee -a /etc/apt/sources.list.d/mono-xamarin.list
echo "deb http://download.mono-project.com/repo/debian wheezy-libjpeg62-compat main" | tee -a /etc/apt/sources.list.d/mono-xamarin.list
apt-get update
apt-get install -y mono-complete
echo "Actually installing nuget"
wget https://dist.nuget.org/win-x86-commandline/latest/nuget.exe
echo "Adding nuget source..."
mono nuget.exe sources add -Name xxx -Source https://ci.appveyor.com/nuget/xxx -Username xxx@example.com -Password -xxx
echo "Listing sources..."
mono nuget.exe list -s xxx
echo "Adding package..."
dotnet add package -s https://ci.appveyor.com/nuget/xxx my.nuget.package

Then I executed the file inside a container:

docker run -v $(pwd)/test.sh:/test.sh --rm microsoft/dotnet:latest /bin/bash test.sh

I got exactly the same results on Linux as I did on my Mac.

Package Source with Name: xxx added successfully.
Listing sources...
Using credentials from config. UserName: xxx@xxx.com
Duration 1.0.1
my.nuget.package 1.0.15
Adding package...
Microsoft (R) Build Engine version 15.1.548.43366
Copyright (C) Microsoft Corporation. All rights reserved.

  Writing /tmp/tmpr9t9l7.tmp
info : Adding PackageReference for package 'my.nuget.package' into project '/TestProject/TestProject.csproj'.
log  : Restoring packages for /TestProject/TestProject.csproj...
info :   GET https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package'
info :   Unauthorized https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package' 915ms
log  : Retrying 'FindPackagesByIdAsyncCore' for source 'https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package''.
log  : Response status code does not indicate success: 401 (Unauthorized).
info :   GET https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package'
info :   Unauthorized https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package' 569ms
log  : Retrying 'FindPackagesByIdAsyncCore' for source 'https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package''.
log  : Response status code does not indicate success: 401 (Unauthorized).
info :   GET https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package'
info :   Unauthorized https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package' 494ms
error: Failed to retrieve information about 'my.nuget.package' from remote source 'https://ci.appveyor.com/nuget/xxx/FindPackagesById()?id='my.nuget.package''.
error:   Response status code does not indicate success: 401 (Unauthorized).

Not sure what the state of your Windows machine is. Maybe the problem is only on Linux and OSX.

[mishra14]

@a-h - Thanks for the script. I will see if I can run stuff on a mac and get a repro.

One thought I had was -just try dotnet add package my.nuget.package ?

If the command is picking up the right config file then it should resolve the feed. That might give us more information.

[mishra14]

@a-h : I did some more digging and found an older issue: #2887

The problem is that dotnet does not support encrypted passwords on non-windows platform yet. mono nuget.exe works because mono seems to support the encryption apis that we use.

So if you do -
mono nuget.exe sources add -Name xxx -Source https://ci.appveyor.com/nuget/xxx -Username xxx@example.com -Password -xxx -StorePasswordInClearText

This is going to put your password into the config file in clear text, so its not the best solution but should unblock you. Let me if that works.

[a-h]

I updated the script to add the -StorePasswordInClearText flag, ran it, and got exactly the same error.

I then tried -StorePasswordInClearText AND replaced dotnet add package -s https://ci.appveyor.com/nuget/xxx my.nuget.package with dotnet add package my.nuget.package. In this case, it simply couldn't find my private package in the public NuGet repo:

Microsoft (R) Build Engine version 15.1.548.43366
Copyright (C) Microsoft Corporation. All rights reserved.

  Writing /tmp/tmpZnd5LG.tmp
info : Adding PackageReference for package 'xxx' into project '/TestProject/TestProject.csproj'.
log  : Restoring packages for /TestProject/TestProject.csproj...
info :   GET https://api.nuget.org/v3-flatcontainer/xxx/index.json
info :   NotFound https://api.nuget.org/v3-flatcontainer/xxx/index.json 139ms
error: Unable to resolve 'xxx' for '.NETStandard,Version=v1.4'.
error: Package 'xxx' is incompatible with 'all' frameworks in project '/TestProject/TestProject.csproj'.

So I'm inclined to think that there's an underlying problem.

[mishra14]

@a-h :

The problem is that mono and dotnet resolve user and machine wide config files differently. When you add source using mono, it goes to the user wide config file that mono uses but dotnet does not know about.

Tested workaround on linux -

What this does is that it adds the source to the local NuGet.config file. This is then picked up by dotnet.

ddcloud@ddanmishr005:~/dev/temp2$ mono nuget.exe sources Add -Name "temp" -Source "https://mishra14.pkgs.visualstudio.com/_packaging/temp/nuget/v2" -username xxx -password xxx -StorePasswordInClearText -configfile ./NuGet.config
Package Source with Name: temp added successfully.
ddcloud@ddanmishr005:~/dev/temp2$ ../temp/dotnet add package testapp
Microsoft (R) Build Engine version 15.2.93.5465
Copyright (C) Microsoft Corporation. All rights reserved.

  Writing /tmp/tmpUvdT4X.tmp
info : Adding PackageReference for package 'testapp' into project '/home/ddcloud/dev/temp2/temp2.csproj'.
log  : Restoring packages for /home/ddcloud/dev/temp2/temp2.csproj...
info :   GET https://dotnetmyget.blob.core.windows.net/artifacts/dotnet-core/nuget/v3/flatcontainer/testapp/index.json
info :   CACHE https://mishra14.pkgs.visualstudio.com/_packaging/temp/nuget/v2/FindPackagesById()?id='testapp'
info :   CACHE https://mishra14.pkgs.visualstudio.com/_packaging/temp/nuget/v2?id=testapp&version=1.0.0
info :   GET https://api.nuget.org/v3-flatcontainer/testapp/index.json
info :   NotFound https://dotnetmyget.blob.core.windows.net/artifacts/dotnet-core/nuget/v3/flatcontainer/testapp/index.json 197ms
info :   NotFound https://api.nuget.org/v3-flatcontainer/testapp/index.json 274ms
info : Package 'testapp' is compatible with all the specified frameworks in project '/home/ddcloud/dev/temp2/temp2.csproj'.
info : PackageReference for package 'testapp' version '1.0.0' updated in file '/home/ddcloud/dev/temp2/temp2.csproj'.

[a-h]

Thanks for the workaround! :) I'll just let my team know that until the dotnet command adds support for adding / restoring packages from private repositories and adding additional package sources, they need to:

• Install Mono
• Install NuGet
• Add private package sources into a local NuGet.config file, making sure to not encrypt their credentials

Has the NuGet team thought of updating the <PackageReference> element of the .csproj format (https://docs.microsoft.com/en-us/dotnet/articles/core/tools/csproj) to include the URL of the package source? Then, when dotnet restore is executed, it would know the correct source to search instead of searching all sources for the package (it looks like a waste of HTTP requests to me).

[mishra14]

@a-h : I have opened a bug to track this request - #5184

[mishra14]

duplicate of #4413. Closing.