Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

curl throws away partial downloads on retry, even if told to resume #1084

Closed
ropery opened this issue Oct 24, 2016 · 2 comments
Closed

curl throws away partial downloads on retry, even if told to resume #1084

ropery opened this issue Oct 24, 2016 · 2 comments
Labels
cmdline tool feature-request

Comments

@ropery
Copy link

ropery commented Oct 24, 2016 •
edited
Loading

Say I want to download a large file, 1 GiB perhaps. I specify -O.
My network connection isn't very reliable, so I specify -m and --retry, so it doesn't hang up.

The problem is, on every retry, the output file is truncated, effectively throwing away all the work so far.

Warning: Transient problem: timeout Will retry in 1 seconds. 5 retries left.
Throwing away 327962289 bytes

This is wasteful, and makes --retry quite useless.
At least, if -C - is specified, it's pretty clear the user wants to always resume automatically.

The workaround is to either not to use --retry & shell-wrap curl in a for loop, or to use wget.

Relevant source, for anyone interested.

curl 7.50.3
@bagder
Copy link
Member

bagder commented Oct 24, 2016

As that output suggests, this is not a bug but actually a missing feature. I would welcome to have such functionality added.

bagder added a commit that referenced this issue Oct 24, 2016
@bagder
TODO: add "--retry should resume"
21b9e54 
See #1084
@bagder
Copy link
Member

bagder commented Oct 27, 2016

This is now mentioned as a suggestion in the TODO. Closing this for now.

@bagder bagder closed this as completed Oct 27, 2016
@bagder bagder mentioned this issue Mar 23, 2018
Closed
agners added a commit to agners/curl that referenced this issue Apr 3, 2018
@agners
tool_operate: resume at last byte when retrying
036ada8 
If continue/resume is enabled, try to also resume when retrying due to
transient connection problems.

Closes curl#1084
@agners agners mentioned this issue Apr 3, 2018
Closed
agners added a commit to agners/curl that referenced this issue Apr 6, 2018
@agners
tool_operate: try resuming downloads when retrying
eafaa04 
If http(s) is used and retry is enabled, try resuming download in case
of transient connection problems.

In case the server reports that range support is not supported, fall
back to the old mode where the file is truncated and download is tried
from start again.

Closes curl#1084
agners added a commit to agners/curl that referenced this issue Apr 6, 2018
@agners
tool_operate: try resuming downloads when retrying
af1d713 
If http(s) is used and retry is enabled, try resuming download in case
of transient connection problems.

In case the server reports that range support is not supported, fall
back to the old mode where the file is truncated and download is tried
from start again.

Closes curl#1084
@lock lock bot locked as resolved and limited conversation to collaborators May 7, 2018
agners added a commit to agners/curl that referenced this issue Nov 7, 2018
@agners
tool_operate: try resuming downloads when retrying
1aae02c 
If http(s) is used and retry is enabled, try resuming download in case
of transient connection problems.

In case the server reports that range support is not supported, fall
back to the old mode where the file is truncated and download is tried
from start again.

Closes curl#1084
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
cmdline tool feature-request
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

tool_operate: resume at last byte when retrying agners/curl
2 participants
@ropery @bagder