Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

curl ignores the Retry-After header with fail-on-error #6408

Closed
bagder opened this issue Jan 4, 2021 · 0 comments
Closed

curl ignores the Retry-After header with fail-on-error #6408

bagder opened this issue Jan 4, 2021 · 0 comments
Assignees

Comments

@bagder
Copy link
Member

@bagder bagder commented Jan 4, 2021

On behalf of @Cherish98 (originally posted here)

I did this

I was trying to download lots of files with curl. Due to rate limit on the server side, I enabled the --retry option so that curl retries when the server returns HTTP 429 Too Many Requests. I also used the fail-on-error (-f, --fail) option to make sure that HTTP error pages are not saved to disk, just in case the last retry still got HTTP 429 or other errors. I think that makes a reasonable use case.

I expected the following

I was expecting curl to use the Retry-After value in the HTTP response for the retry delay period. However, curl (libcurl to be precise, but this issue is more concerned with the tool, as library users can deal with errors programmatically) ignores (almost) all the HTTP headers with the CURLOPT_FAILONERROR option. I would suggest that curl looks for the Retry-After header in case of HTTP error 429, even the request itself is marked as failed.

curl/libcurl version

curl master

operating system

any

bagder added a commit that referenced this issue Jan 4, 2021
... so that Retry-After and other meta-content can still be used.

Added 1634 to verify. Adjusted test 194 and 281 since --fail now also
includes the header-terminating CRLF in the output before it exits.

Fixes #6408
@bagder bagder self-assigned this Jan 4, 2021
@bagder bagder closed this in ab525c0 Jan 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

1 participant