UseDotNet@2 task failed on 404 at least retry expected

[elias-solutions]

Note

Issues in this repo are for tracking bugs, feature requests and questions for the tasks in this repo

For a list:
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks

If you have an issue or request for the Azure Pipelines service, use developer community instead:

https://developercommunity.visualstudio.com/spaces/21/index.html )

Required Information

Entering this information will route you directly to the right team and expedite traction.

Question, Bug, or Feature?
Type: Bug

Enter Task Name: UseDotNet@2

list here (V# not needed):
https://github.com/Microsoft/azure-pipelines-tasks/tree/master/Tasks

Environment

• Server - Azure Pipelines or TFS on-premises?

  • If using TFS on-premises, provide the version:

  • If using Azure Pipelines, provide the account name, team project name, build definition name/build number:

• Agent - Hosted or Private:

  • If using Hosted agent, provide agent queue name:

  • If using private agent, provide the OS of the machine running the agent and the agent version:

Issue Description

UseDotNet@2 failed with a status code 404 while downloading: https://download.visualstudio.microsoft.com/download/pr/5f0f07ab-cd9a-4498-a9f7-67d90d582180/2a3db6698751e6cbb93ec244cb81cc5f/dotnet-sdk-5.0.202-linux-x64.tar.gz

in Task in our yml:

This causes in almost every second build.

Suggestion Retry expected:

• Project azure-pipelines-tool-lib
• in file tool.ts
• in function downloadTool()
• line number (231)

Task logs

[Enable debug logging and please provide the zip file containing all the logs for a speedy resolution]

Troubleshooting

Checkout how to troubleshoot failures and collect debug logs: https://docs.microsoft.com/en-us/vsts/build-release/actions/troubleshooting

Error logs

Similar Issue

#14793

[llehn]

We are having the same problem with 3.1.411:

##[error]Failed` while installing version: 3.1.411 at path: /opt/hostedtoolcache/dotnet with error: Could not download installation package from this URL: https://download.visualstudio.microsoft.com/download/pr/c7afc344-acf0-4a6c-8e53-4455f2bcfc85/321418f6300e475f6c5d52849efa2e2e/dotnet-sdk-3.1.411-linux-x64.tar.gz Error: Error: Unexpected HTTP response: 404

It appears quite frequently and breaks many of our builds.

[jeromelaban]

I've opened the issue here as well: https://developercommunity.visualstudio.com/t/Use-NET-Core-task-fails-randomly-to-i/1486997.

Feels like a CDN issue as the URLs sometimes work.

[arroyc]

Having this issue randomly since last few hours for 3.1.411, retry doesn't work.

Starting: Use .NET Core sdk 3.1.x

Task : Use .NET Core
Description : Acquires a specific version of the .NET Core SDK from the internet or the local cache and adds it to the PATH. Use this task to change the version of .NET Core used in subsequent tasks. Additionally provides proxy support.
Version : 2.184.0
Author : Microsoft Corporation
Help : https://aka.ms/AA4xgy0

Tool to install: .NET Core sdk version 3.1.x.
Found version 3.1.411 in channel 3.1 for user specified version spec: 3.1.x
get-os-distro: Error: Distribution specific OS name and version could not be detected: UName = Linux
Version 3.1.411 was not found in cache.
Getting URL to download .NET Core sdk version: 3.1.411.
Detecting OS platform to find correct download package for the OS.
/home/vsts/work/_tasks/UseDotNet_b0ce7256-7898-45d3-9cb5-176b752bfea6/2.184.0/externals/get-os-distro.sh
Primary:linux-x64
Legacy:-x64
Detected platform (Primary): linux-x64
Detected platform (Legacy): -x64
Downloading: https://download.visualstudio.microsoft.com/download/pr/c7afc344-acf0-4a6c-8e53-4455f2bcfc85/321418f6300e475f6c5d52849efa2e2e/dotnet-sdk-3.1.411-linux-x64.tar.gz
##[error]Failed while installing version: 3.1.411 at path: /opt/hostedtoolcache/dotnet with error: Could not download installation package from this URL: https://download.visualstudio.microsoft.com/download/pr/c7afc344-acf0-4a6c-8e53-4455f2bcfc85/321418f6300e475f6c5d52849efa2e2e/dotnet-sdk-3.1.411-linux-x64.tar.gz Error: Error: Unexpected HTTP response: 404
Finishing: Use .NET Core sdk 3.1.x

[arroyc]

Pool: Azure Pipelines
Image: ubuntu-latest
Agent: Hosted Agent
Started: Today at 9:32 AM
Duration: 4m 2s

[tongyuze]

I'm seeing the same issue here.
##[error]Failed while installing version: 3.1.17 at path: /opt/hostedtoolcache/dotnet with error: Could not download installation package from this URL: https://download.visualstudio.microsoft.com/download/pr/7a4bfa73-e500-45aa-8e10-04dc4910d8ae/f8b74c973752d3ea095fe06aa625f7f7/dotnet-runtime-3.1.17-linux-x64.tar.gz Error: Error: Unexpected HTTP response: 404

[bishal-pdMSFT]

This seems to be an issue with the download links provided in index file for downloading .NET. I have created an issue on .NET repo.

[davesmits]

azure devops team seems also planning something or the retry: https://dev.azure.com/mseng/AzureDevOpsRoadmap/_workitems/edit/1859330/

[arroyc]

@davesmits @bishal-pdMSFT any workaround?

[bishal-pdMSFT]

@arroyc as a workaround, you can try running the dotnet-install which is provided by .NET team. For example:
dotnet-install.ps1 -Version 3.1.411

You could run this script in a Powershell task and replace the UseDotNet task for the time being.
Please note that you might have to add the directory where the SDK was installed to PATH variable so that subsequent DotNetCLI task can find the installed version.

[rp-bneagle]

@davesmits - the link you provided seems to be locked/hidden. Is there a public link?

[davesmits]

@rp-bneagle try in-private. then it mostly works

[EnmanuelParedesR]

I'm having the same issue here.

2021-07-26T20:40:34.1723694Z ##[section]Starting: Using .NET Core 3.1.x
2021-07-26T20:40:34.1735438Z ==============================================================================
2021-07-26T20:40:34.1735747Z Task         : Use .NET Core
2021-07-26T20:40:34.1736200Z Description  : Acquires a specific version of the .NET Core SDK from the internet or the local cache and adds it to the PATH. Use this task to change the version of .NET Core used in subsequent tasks. Additionally provides proxy support.
2021-07-26T20:40:34.1736661Z Version      : 2.184.0
2021-07-26T20:40:34.1736853Z Author       : Microsoft Corporation
2021-07-26T20:40:34.1737087Z Help         : https://aka.ms/AA4xgy0
2021-07-26T20:40:34.1737337Z ==============================================================================
2021-07-26T20:40:34.5712345Z Tool to install: .NET Core sdk version 3.1.x.
2021-07-26T20:40:35.6191715Z Found version 3.1.411 in channel 3.1 for user specified version spec: 3.1.x
2021-07-26T20:40:35.6716005Z get-os-distro: Error: Distribution specific OS name and version could not be detected: UName = Linux
2021-07-26T20:40:35.6719898Z Version 3.1.411 was not found in cache.
2021-07-26T20:40:35.6722478Z Getting URL to download .NET Core sdk version: 3.1.411.
2021-07-26T20:40:35.6723039Z Detecting OS platform to find correct download package for the OS.
2021-07-26T20:40:35.6726643Z [command]/home/vsts/work/_tasks/UseDotNet_b0ce7256-7898-45d3-9cb5-176b752bfea6/2.184.0/externals/get-os-distro.sh
2021-07-26T20:40:35.6727250Z Primary:linux-x64
2021-07-26T20:40:35.6727647Z Legacy:-x64
2021-07-26T20:40:35.6728132Z Detected platform (Primary): linux-x64
2021-07-26T20:40:35.6729063Z Detected platform (Legacy): -x64
2021-07-26T20:40:35.6732145Z Downloading: https://download.visualstudio.microsoft.com/download/pr/c7afc344-acf0-4a6c-8e53-4455f2bcfc85/321418f6300e475f6c5d52849efa2e2e/dotnet-sdk-3.1.411-linux-x64.tar.gz
2021-07-26T20:40:35.6779043Z ##[error]Failed while installing version: 3.1.411 at path: /opt/hostedtoolcache/dotnet with error: Could not download installation package from this URL: https://download.visualstudio.microsoft.com/download/pr/c7afc344-acf0-4a6c-8e53-4455f2bcfc85/321418f6300e475f6c5d52849efa2e2e/dotnet-sdk-3.1.411-linux-x64.tar.gz Error: Error: Unexpected HTTP response: 404
2021-07-26T20:40:35.6808014Z ##[section]Finishing: Using .NET Core 3.1.x
{"mode":"full","isActive":false}

[llehn]

This is now breaking more than 50% of our builds, SEVERELY impacting development. Please fix urgently!

[a-nightingale]

This had been intermittent and re-queue of pipeline would resolve. Since 3:00 PM Eastern, we have what looks like 100% fail rate on all .NET Core 3.1 pipelines downloading.

Sample log:

2021-07-26T21:24:49.3960942Z Tool to install: .NET Core sdk version 3.1.100.
2021-07-26T21:24:50.3796282Z Found version 3.1.100 in channel 3.1 for user specified version spec: 3.1.100
2021-07-26T21:24:51.3634256Z Version 3.1.100 was not found in cache.
2021-07-26T21:24:51.3635186Z Getting URL to download .NET Core sdk version: 3.1.100.
2021-07-26T21:24:51.3636063Z Detecting OS platform to find correct download package for the OS.
2021-07-26T21:24:51.3643402Z [command]/agent/_work/_tasks/UseDotNet_b0ce7256-7898-45d3-9cb5-176b752bfea6/2.184.0/externals/get-os-distro.sh
2021-07-26T21:24:51.3644746Z Primary:linux-x64
2021-07-26T21:24:51.3645781Z Legacy:ubuntu.18.04-x64
2021-07-26T21:24:51.3647126Z Detected platform (Primary): linux-x64
2021-07-26T21:24:51.3648394Z Detected platform (Legacy): ubuntu.18.04-x64
2021-07-26T21:24:51.3653901Z Downloading: https://download.visualstudio.microsoft.com/download/pr/d731f991-8e68-4c7c-8ea0-fad5605b077a/49497b5420eecbd905158d86d738af64/dotnet-sdk-3.1.100-linux-x64.tar.gz
2021-07-26T21:24:51.3692122Z ##[error]Failed while installing version: 3.1.100 at path: /agent/_work/_tool/dotnet with error: Could not download installation package from this URL: https://download.visualstudio.microsoft.com/download/pr/d731f991-8e68-4c7c-8ea0-fad5605b077a/49497b5420eecbd905158d86d738af64/dotnet-sdk-3.1.100-linux-x64.tar.gz Error: Error: Unexpected HTTP response: 404
2021-07-26T21:24:51.3722129Z ##[section]Finishing: Use .NET Core sdk

[arroyc]

guys .. sorry but its not intermittent any more for me .. im seeing 100% failure in first time and repeats ... please help !

[arroyc]

can we create a backup cdn or something like that?

[SamuelGarrattIqa]

My team was using the ADO ubuntu agent https://github.com/actions/virtual-environments/blob/ubuntu20/20210718.1/images/linux/Ubuntu2004-README.md which already had dotnet installed.
I removed that task (as it was not needed anyone) and everything still worked as before

[mikewop]

Failing also for .Net 5
##[error]Failed while installing version: 5.0.302 at path: /opt/hostedtoolcache/dotnet with error: Could not download installation package from this URL: https://download.visualstudio.microsoft.com/download/pr/8468e541-a99a-4191-8470-654fa0747a9a/cb32548d2fd3d60ef3fe8fc80cd735ef/dotnet-sdk-5.0.302-linux-x64.tar.gz Error: Error: Unexpected HTTP response: 404

[bishal-pdMSFT]

@llehn @a-nightingale @arroyc the issue with outside of UseDotNet task and hence really difficult to fix inside the task.

You can use a similar to following powershell task as a workaround replacement

- powershell: |
   
   Invoke-WebRequest 'https://dot.net/v1/dotnet-install.ps1' -OutFile 'dotnet-install.ps1'
   ./dotnet-install.ps1 -InstallDir '$(Agent.ToolsDirectory)/dotnet' -Version '3.1.411' 
   Write-Host '##vso[task.prependpath]$(Agent.ToolsDirectory)/dotnet'
  displayName: 'Install .NET workaround'

[bishal-pdMSFT]

My team was using the ADO ubuntu agent https://github.com/actions/virtual-environments/blob/ubuntu20/20210718.1/images/linux/Ubuntu2004-README.md which already had dotnet installed.
I removed that task (as it was not needed anyone) and everything still worked as before

This is also a very reasonable workaround. Microsoft hosted agents already have many version of .NET installed and hence you can re-try with disabling the UseDotNet task. Should work for a majority of cases.

In case you are using self hosted agents, then you can pre-install required version of .NET on the agent machine and remove dependency on the task.

[EnmanuelParedesR]

Right now I tried again and one of my pipelines went ok with the step. I don't know if is fixed or is just the intermittence.

[mairaw]

Folks, a fix was deployed around an hour ago and we wanna confirm that customers are not hitting the issue any longer. Can you let me know if your pipeline is now working or still doesn't? Thanks!

[isra-fel]

It works for me! No issues in the last hour 👍