Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error getting zip file of "site/wwwroot": Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. #1448

Open
miparnisari opened this issue Jan 29, 2015 · 12 comments

Comments

@miparnisari
Copy link

@miparnisari miparnisari commented Jan 29, 2015

I'm getting this random error when downloading the ZIP file corresponding to "site/wwwroot". When the download succeeds, the file is around 20 MB.

This is the code I'm using:
$apiUrl = "https://$websiteName.scm.azurewebsites.net/api/zip/site/wwwroot/"
$method = "GET"
$result = Invoke-RestMethod -Uri $apiUrl -Headers @{Authorization=("Basic {0}" -f $base64AuthInfo)} -Method $method -OutFile "wwwroot.zip" -TimeoutSec 600

The error is "Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host."

@davidebbo
Copy link
Member

@davidebbo davidebbo commented Jan 29, 2015

As a test, can you please try the following:

  • Go to the Kudu Console
  • Go into the site folder
  • Click the download link next to the wwwroot folder

This should invoke basically the same zip API that you are using from PowerShell. It will be interesting to see if you get the same issue.

@miparnisari
Copy link
Author

@miparnisari miparnisari commented Jan 30, 2015

Yes, that works. I don't have any issues when downloading from a browser, it's just when using PowerShell.

I tried setting the user agent to [Microsoft.PowerShell.Commands.PSUserAgent]::Chrome but I'm getting the same error.

@davidebbo
Copy link
Member

@davidebbo davidebbo commented Jan 31, 2015

Strange that it would only happen from PowerShell. I usually do this using curl, e.g.

curl -o foo.zip %KUDU_URL%/api/zip/site/wwwroot/

Maybe worth trying. If that works reliably, then we'll have to blame PowerShell! :)

@AWahlqvist
Copy link

@AWahlqvist AWahlqvist commented Oct 14, 2016

Found this issue through Google, having the same problem. It seems to happen when the site contains a lot of files, at least in my case.

It looks like it's tied to Invoke-RestMethod and Invoke-WebRequest though, it seems to work fine when using System.Net.WebClient instead, so that can be a workaround for this.

Basically:
`
$WebClient = New-Object System.Net.WebClient
$WebClient.Headers.Add('Authorization', 'Base 64 auth string...')
$WebClient.Headers.Add('ContentType', 'multipart/form-data')

$WebClient.DownloadFile($Uri, $OutFile)
`

@davidebbo
Copy link
Member

@davidebbo davidebbo commented Oct 14, 2016

That's odd. If it works reliably from the browser and using curl (as in the real curl, not the PowerShell alias), then I'll just blame it on something PowerShell is doing!

@AWahlqvist
Copy link

@AWahlqvist AWahlqvist commented Oct 14, 2016

I tend to agree, the issue seems to exist in the built-in cmdlets Invoke-RestMethod and Invoke-WebRequest, not the Kudu API. Just added that here if someone else runs into this issue so they can see the workaround above that seems to run just fine.

But maybe this issue should be closed and a new one added to the PowerShell repo instead? I can add one there.

@davidebbo
Copy link
Member

@davidebbo davidebbo commented Oct 14, 2016

Yes, probably. However, to make it more convincing, it would be nice to have a repro of the PowerShell issue that doesn't involve Kudu. e.g. if you host the same zip somewhere else, does it happen with PowerShell? It could be that what makes it Kudu specific is that it can take a while for the bits to start flowing, as the zip has to first be created server side, and maybe that throws off PowerShell.

@AWahlqvist
Copy link

@AWahlqvist AWahlqvist commented Oct 14, 2016

I'm pretty sure it wouldn't if the zip file was available already. What I've noticed is that the download slows down significantly when a folder contains a large number of files (I'm guessing it gets "streamed" while it's being packed?), and it's during this phase those cmdlets throws this error. So the download nearly finishes sometimes before this occurs.

This specific error usually means that the remote host actually sent a tcp RST to the client though, which would be a bit odd in this case?

@davidebbo
Copy link
Member

@davidebbo davidebbo commented Oct 14, 2016

Not sure. We just use System.IO.Compression.ZipArchive and System.Net.Http.PushStreamContent under the cover (see code here and here). And the file system is slower than a typical file system, so zipping a large tree is pretty slow.

@sanjay3290
Copy link

@sanjay3290 sanjay3290 commented Sep 28, 2020

its 2020 and i still face the same issue. Did somebody found a workaround?

@AWahlqvist
Copy link

@AWahlqvist AWahlqvist commented Sep 29, 2020

@sanjay3290 : Not really, but we started to upload a PowerShell script that zips up the folder instead that also supports running "async" from the initial call so it doesn't get interrupted when the Azure front end cuts off the connection after 230 seconds. That has worked well for us, but not sure if it qualifies as a workaround.

@ericthomas1
Copy link

@ericthomas1 ericthomas1 commented Apr 29, 2021

2021 here...

I don't see a Site option in the Kudo console anymore. How do I download a .zip from here?

image

Also, hitting https://$FunctionAppName.scm.azurewebsites.net/api/zip/site/wwwroot/ yields a .zip file with only the Function's host.json file, no source code or support files.

image

How does one download the .zip for a Python Function!? (please don't say "FTP")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants