Exceeding maxfilesize once breaks future downloads on same http/2 connection #3392
Labels
Comments
|
Here's a much smaller standalone program that reproduces the problem: #include <curl/curl.h>
int main(void) {
curl_global_init(CURL_GLOBAL_ALL);
CURL* curl = curl_easy_init();
FILE* devnull = fopen("/dev/null", "w");
curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);
curl_easy_setopt(curl, CURLOPT_LOW_SPEED_LIMIT, 1L);
curl_easy_setopt(curl, CURLOPT_LOW_SPEED_TIME, 5L);
/* remote returns 200 */
curl_easy_setopt(curl, CURLOPT_URL, "https://mirror.ams1.nl.leaseweb.net/archlinux/core/os/x86_64/core.db");
curl_easy_setopt(curl, CURLOPT_WRITEDATA, devnull);
curl_easy_perform(curl);
/* remote returns 404, but response body exceeds max filesize */
curl_easy_setopt(curl, CURLOPT_URL, "https://mirror.ams1.nl.leaseweb.net/archlinux/core/os/x86_64/core.db.sig");
curl_easy_setopt(curl, CURLOPT_MAXFILESIZE_LARGE, 16384L);
curl_easy_perform(curl);
/* this hangs, eventually times out due to low speed limit */
curl_easy_setopt(curl, CURLOPT_URL, "https://mirror.ams1.nl.leaseweb.net/archlinux/extra/os/x86_64/extra.db");
curl_easy_perform(curl);
/* this hangs, eventually times out due to low speed limit */
curl_easy_setopt(curl, CURLOPT_URL, "https://mirror.ams1.nl.leaseweb.net/archlinux/extra/os/x86_64/extra.db.sig");
curl_easy_perform(curl);
curl_easy_cleanup(curl);
fclose(devnull);
return 0;
}Amending this program to include: curl_easy_setopt(curl, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);...makes the program work as expected. |
bagder
added a commit
that referenced
this issue
Dec 20, 2018
Reported-by: Florian Pritz Fixes #3392
|
With the patch in #3399 it seems the problem is fixed for me. Would appreciate if you verify how it behaves for you! |
|
#3399 fixes the problem with pacman for me too. Thanks! |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
When using pacman (which uses libcurl for downloading) with an HTTP/2 server, further download requests break when the server returns a response that is bigger than the maxfilesize set via CURLOPT_MAXFILESIZE_LARGE. libcurl returns the error to pacman as seen in the log below (look for "Maximum file size exceeded"), but when pacman tries to download another file over the same connection, the connection doesn't appear to work correctly any more. The curl docs say that applications should continue to work unmodified so I'm posting this issue here before contacting the pacman developers.
Edit: The filesize exceed error is expected (kind of). Arch Linux does not sign their databases, but pacman supports signed databases and so tries to download a signature file. Since databases are not signed, the signature does not exist and a 404 error is returned. In case of one particular server, the 404 error page is bigger than our max file size for signatures and thus triggers this error.
When the server only supports HTTP/1.1, pacman works correctly.
When trying to reproduce the problem with
curl --max-filesize 16384 -sv https://mirror.ams1.nl.leaseweb.net/archlinux/staging/os/x86_64/staging.db{.sig,} -o /dev/nullI can't seem to trigger the same issue.You can view pacman's download code here. I've modified it locally to enable CURLOPT_VERBOSE to get the output shown below.
curl/libcurl version
curl 7.62.0 (x86_64-pc-linux-gnu) libcurl/7.62.0 OpenSSL/1.1.1a zlib/1.2.11 libidn2/2.0.5 libpsl/0.20.2 (+libidn2/2.0.4) libssh2/1.8.0 nghttp2/1.34.0
Release-Date: 2018-10-31
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL
operating system
Arch Linux
The text was updated successfully, but these errors were encountered: