8.1.0 download slower than 8.0.1

[pandada8]

I did this

When download from same url with same network, curl 8.1.0 downloads half as fast as curl 8.0.1

pan@panda-router ~ ❯❯❯ curl 'https://mirrors.tuna.tsinghua.edu.cn/archlinux/iso/latest/archlinux-2023.05.03-x86_64.iso' -o /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  813M  100  813M    0     0  94.2M      0  0:00:08  0:00:08 --:--:-- 95.8M
pan@panda-router ~ ❯❯❯ curl 'https://mirrors.tuna.tsinghua.edu.cn/archlinux/iso/latest/archlinux-2023.05.03-x86_64.iso' -o /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  813M  100  813M    0     0  45.4M      0  0:00:17  0:00:17 --:--:-- 47.0M

(try a nearby mirror to test if you want to replicate, the provided link speed may be not optimal in your location)

I also tried different endpoint in a different location (vultr vps in la). It seems the slowdown ratio may be different, but 8.1.0 IS slower than 8.0.1

I expected the following

download speed shouldn't make any difference

curl/libcurl version

# curl -V
curl 8.1.0 (x86_64-pc-linux-gnu) libcurl/8.1.0 OpenSSL/3.0.8 zlib/1.2.13 brotli/1.0.9 zstd/1.5.5 libidn2/2.3.4 libpsl/0.21.2 (+libidn2/2.3.4) libssh2/1.10.0 nghttp2/1.53.0
Release-Date: 2023-05-17
Protocols: dict file ftp ftps gopher gophers http https imap imaps mqtt pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz NTLM NTLM_WB PSL SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd

I did some bisect, it seems the slowdown is introduced by 744dcf2
Indeed, the provided link is http2 enabled

operating system

Linux panda-router 6.2.11-arch1-1 #1 SMP PREEMPT_DYNAMIC Thu, 13 Apr 2023 16:59:24 +0000 x86_64 GNU/Linux

[bagder]

@pandada8 what's the average ping time client <=> server in this setup ?

[pandada8]

Around 3.5ms

[icing]

This seems to be the effect of the lower stream window size we introduced. I have a rather slow downlink, but I can see that the higher the ping time to a mirror, the less it becomes saturated. Increasing the http/2 window size - in the source - helps.

The relevant place in lib/http2.cis:

#define H2_STREAM_WINDOW_SIZE   (512 * 1024)

I would be interested to know what you experience when you increase that to 5 * 1024 * 1024. I assume you can build curl?

[pandada8]

I play around with different values, given 3.5ms latency

H2_STREAM_WINDOW_SIZE	Download Speed
512 * 1024	50MB/s
5 * 512 * 1024	77MB/s
5 * 1024 * 1024	84MB/s
15 * 1024 * 1024	90.3MB/s
--http1.1	91.8MB/s

It does improve the performance, the final difference could be a margin of error

[icing]

Thanks for the measurements. Since I want a quick fix for the upcoming patch release, I suggest I settle for a 10MB window for now.

There might be a good way to adjust that more dynamically, but that is outside the current time frame.

[icing]

PR #11167 in CI now.