NuGet can timeout on slow disks

[victorhurdugaci]

We've seen this on our CI. @joelverhagen and I investigated.

My package folder was quite large (9ish GB) and I was running on a D3 Azure VM. dotnet restore (which calls in NuGet) was timing out, probably because of all the disk IO. Once we cleared the packages folder, everything started working.

cc @yishaigalatzer

[joelverhagen]

Although disk may be the sole cause, I would also like to eliminate the possibility that there were too many parallel HTTP requests.

This evidence is certainly anecdotal, but I noticed when we were debugging together that there was a ton of hanging HTTP requests. This could be related to the fact that we do not have ServicePointManager.DefaultConnectionLimit on dotnet CLI (since .NET Core does not have a `ServicePointManager). This needs investigation.

[joelverhagen]

I've so far been unsuccessful in reproducing the problem.

Constants throughout my investigation:

• Restored aspnet/Common
• Used Windows Server 2008 R2
• Used an Azure D VM
• VM was in US West

> dotnet --version

.NET Command Line Tools (1.0.0-beta-001496)

Product Information:
 Version:     1.0.0-beta-001496
 Commit Sha:  962411969e

Runtime Environment:
 OS Name:     Windows
 OS Version:  6.1.7601
 OS Platform: Windows
 Runtime Id:  win7-x64

Different conditions tried for dotnet restore:

• Azure D1 and D3
• Huge packages folder (everyone on aspnetcidev MyGet and dotnet MyGet)
• Huge v3-cache folder
• While having a lot of network and disk load (generated via a custom tool)
• Restore a bigger repository (dotnet/roslyn, dotnet/coreclr, dotnet/corefx)

I was not able to get any hangs.

[yishaigalatzer]

You also need the following scenarios to try

D2 vm
Large packages folder with lots of versions and lots of packages

[joelverhagen]

@yishaigalatzer, thanks for the suggestions. I've tried both still with no dice. D1, D2, and D3 all seems to act the same.

The packages folder has this topology:

Total package IDs: 947
Total package versions: 11536

Min version count: 1
Avg version count: 12.181626187962
Max version count: 32

Histogram:
    1 -   4 versions: 154 IDs
    4 -   7 versions: 417 IDs
    7 -  10 versions:  15 IDs
   10 -  13 versions:   2 IDs
   13 -  16 versions:   3 IDs
   16 -  20 versions:  48 IDs
   20 -  23 versions:  17 IDs
   23 -  26 versions: 194 IDs
   26 -  29 versions:  77 IDs
   29 -  32 versions:  20 IDs

The packages folder is around 20 GB.

[yishaigalatzer]

What happens when you have 400 versions (imagine a CI building kicking off hourly leading to many versions daily

[joelverhagen]

I make sure 500 versions of RavenDB.Client were in the packages folder then added that as a dependency to one of the projects. Still no repro.

[victorhurdugaci]

Do you restore through nuget or dotnet restore?

On Wed, Mar 2, 2016 at 9:45 AM -0800, "Joel Verhagen" <notifications@github.commailto:notifications@github.com> wrote:

I make sure 500 versions of RavenDB.Client were in the packages folder then added that as a dependency to one of the projects. Still no repro.

Reply to this email directly or view it on GitHubhttps://github.com//issues/2201#issuecomment-191345802.

[joelverhagen]

I am only using nuget.exe to populate the packages folder via packages.config. I am using dotnet restore for the restore.

[joelverhagen]

Additionally, I am running git clean -xfd before every restore to make sure old lock files don't effect the results.

[yishaigalatzer]

@joelverhagen this looks related to the timeout/retry issue we discussed on Friday, please dedup/close as necessary

[joelverhagen]

This is fixed with a change to add timeout to downloads (fail is no bytes received from the network in X seconds).
Duplicate of #2269