Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

curl throws away partial downloads on retry, even if told to resume #1084

Closed
ropery opened this issue Oct 24, 2016 · 2 comments
Closed

curl throws away partial downloads on retry, even if told to resume #1084

ropery opened this issue Oct 24, 2016 · 2 comments

Comments

@ropery
Copy link

ropery commented Oct 24, 2016

Say I want to download a large file, 1 GiB perhaps. I specify -O.
My network connection isn't very reliable, so I specify -m and --retry, so it doesn't hang up.

The problem is, on every retry, the output file is truncated, effectively throwing away all the work so far.

Warning: Transient problem: timeout Will retry in 1 seconds. 5 retries left.
Throwing away 327962289 bytes

This is wasteful, and makes --retry quite useless.
At least, if -C - is specified, it's pretty clear the user wants to always resume automatically.

The workaround is to either not to use --retry & shell-wrap curl in a for loop, or to use wget.

Relevant source, for anyone interested.


curl 7.50.3

@bagder
Copy link
Member

bagder commented Oct 24, 2016

As that output suggests, this is not a bug but actually a missing feature. I would welcome to have such functionality added.

bagder added a commit that referenced this issue Oct 24, 2016
@bagder
Copy link
Member

bagder commented Oct 27, 2016

This is now mentioned as a suggestion in the TODO. Closing this for now.

@bagder bagder closed this as completed Oct 27, 2016
agners added a commit to agners/curl that referenced this issue Apr 3, 2018
If continue/resume is enabled, try to also resume when retrying due to
transient connection problems.

Closes curl#1084
agners added a commit to agners/curl that referenced this issue Apr 6, 2018
If http(s) is used and retry is enabled, try resuming download in case
of transient connection problems.

In case the server reports that range support is not supported, fall
back to the old mode where the file is truncated and download is tried
from start again.

Closes curl#1084
agners added a commit to agners/curl that referenced this issue Apr 6, 2018
If http(s) is used and retry is enabled, try resuming download in case
of transient connection problems.

In case the server reports that range support is not supported, fall
back to the old mode where the file is truncated and download is tried
from start again.

Closes curl#1084
@lock lock bot locked as resolved and limited conversation to collaborators May 7, 2018
agners added a commit to agners/curl that referenced this issue Nov 7, 2018
If http(s) is used and retry is enabled, try resuming download in case
of transient connection problems.

In case the server reports that range support is not supported, fall
back to the old mode where the file is truncated and download is tried
from start again.

Closes curl#1084
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Development

Successfully merging a pull request may close this issue.

2 participants