I was trying to download lots of files with curl. Due to rate limit on the server side, I enabled the --retry option so that curl retries when the server returns HTTP 429 Too Many Requests. I also used the fail-on-error (-f, --fail) option to make sure that HTTP error pages are not saved to disk, just in case the last retry still got HTTP 429 or other errors. I think that makes a reasonable use case.
I expected the following
I was expecting curl to use the Retry-After value in the HTTP response for the retry delay period. However, curl (libcurl to be precise, but this issue is more concerned with the tool, as library users can deal with errors programmatically) ignores (almost) all the HTTP headers with the CURLOPT_FAILONERROR option. I would suggest that curl looks for the Retry-After header in case of HTTP error 429, even the request itself is marked as failed.
The text was updated successfully, but these errors were encountered:
... so that Retry-After and other meta-content can still be used.
Added 1634 to verify. Adjusted test 194 and 281 since --fail now also
includes the header-terminating CRLF in the output before it exits.