Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

http fails on 416 when resuming #10521

Smackd0wn opened this issue Feb 15, 2023 · 2 comments

http fails on 416 when resuming #10521

Smackd0wn opened this issue Feb 15, 2023 · 2 comments


Copy link

Smackd0wn commented Feb 15, 2023

I did this

On git master, curl --fail -O -C - http://host/file when the local file had already been completely downloaded.

Server response:

< HTTP/1.1 416 Requested Range Not Satisfiable
< Server: nginx/1.22.1
< Date: Wed, 15 Feb 2023 13:29:29 GMT
< Content-Type: text/html; charset=utf8
< Content-Length: 197
< Content-Range: bytes */1234567

I expected the following

curl exits with code 0 (no error, even with --fail), because in http_should_fail():


Lines 1161 to 1167 in 3027611

** A 416 response to a resume request is presumably because the file is
** already completely downloaded and thus not actually a fail.
if(data->state.resume_from && data->state.httpreq == HTTPREQ_GET &&
httpcode == 416)
return FALSE;

However, when the requested range was unsatisfied (e.g., Content-Range: */1234567), data->state.resume_from is overwritten with 0 (see line 3479), and thus the check above always fail.


Lines 3452 to 3480 in 3027611

else if(!k->http_bodyless && checkprefix("Content-Range:", headp)) {
/* Content-Range: bytes [num]-
Content-Range: bytes: [num]-
Content-Range: [num]-
Content-Range: [asterisk]/[total]
The second format was added since Sun's webserver
JavaWebServer/1.1.1 obviously sends the header this way!
The third added since some servers use that!
The fourth means the requested range was unsatisfied.
char *ptr = headp + strlen("Content-Range:");
/* Move forward until first digit or asterisk */
while(*ptr && !ISDIGIT(*ptr) && *ptr != '*')
/* if it truly stopped on a digit */
if(ISDIGIT(*ptr)) {
if(!curlx_strtoofft(ptr, NULL, 10, &k->offset)) {
if(data->state.resume_from == k->offset)
/* we asked for a resume and we got it */
k->content_range = TRUE;
data->state.resume_from = 0; /* get everything */

@vszakats vszakats added the HTTP label Feb 16, 2023
Copy link

piru commented Feb 21, 2023

It appears nginx returns: content-range: bytes */<filesize> in this case. For example Apache2 doesn't include the content-range header for 416 response.

This seems to fix this issue, but I am not entirely sure if this is the best way to fix it:

diff --git a/lib/http.c b/lib/http.c
index ee31aa2c6..caa3d50a1 100644
--- a/lib/http.c
+++ b/lib/http.c
@@ -3476,7 +3476,7 @@ CURLcode Curl_http_header(struct Curl_easy *data, struct connectdata *conn,
           k->content_range = TRUE;
-    else
+    else if(416 != k->httpcode)
       data->state.resume_from = 0; /* get everything */

Comments are welcome.

piru pushed a commit to piru/curl that referenced this issue Feb 28, 2023
416 Requested Range Not Satisfiable returned by some HTTP daemons
(nginx at least) appear to include Content-Range header. This
confuses curl and makes it think that the whole file needs to be
downloaded again, resulting in download failure when in fact the
while file has already been downloaded.

This change makes curl disregard Content-Range header on a 416
response, allowing other existing logic consider the file fully

Fixes curl#10521
Copy link

hmeine commented Aug 9, 2023

I run into this problem as well. I tried running curl -C - to download a build artifact from an artifactory server, hoping that repeated script runs would not download the large file multiple times.

The server replies with

HTTP/1.1 416 Requested Range Not Satisfiable
Connection: keep-alive
Content-Length: 0
Content-Range: bytes */977880575
Date: Wed, 09 Aug 2023 09:57:07 GMT
Server: nginx/1.18.0 (Ubuntu)
X-Artifactory-Id: ***redacted***
X-Artifactory-Node-Id: ***redacted***
X-Jfrog-Version: Artifactory/7.55.13 75513900

and curl fails (exit code 22).

As I am not familiar with the curl codebase, I have problems reviewing the change in #10644 and do not yet understand what is the problem with it.

@bagder bagder closed this as completed in 225db91 Oct 30, 2023
zuoxiaofeng pushed a commit to zuoxiaofeng/curl that referenced this issue Nov 28, 2023
Finding a 'Content-Range:' in the response changed the handling.

Add test case 1475 to verify -C - with 416 and Content-Range: header,
which is almost exactly like test 194 which instead uses a fixed -C
offset. Adjusted test 194 to also be considered fine.

Fixes curl#10521
Reported-by: Smackd0wn
Fixes curl#12174
Reported-by: Anubhav Rai
Closes curl#12176
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Successfully merging a pull request may close this issue.

4 participants