curl ngtcp performance regression after rate limit redesign

[koujaz]

I did this

Starting w/ commit 24b36fd we see significant performance regression (up to 1000%) when trying to download larger files (1MB+) on links w/ higher throughput (>200MBps) with lower RTT (>30ms).

For reference download parameters (10MB file over 250Mbs link w/ 30ms RTT, no loss, no jitter) let's compare build from previous commit (bfde781) and the affected one. Here are key differences:

• overall download time: 0.6s -> 6s
• number of packets in pcap: 9700 -> 12000
• number of MSD frames sent by client (_ws.col.info ~ "MSD"): 35 -> 550
• number of SDB frames sent by server (_ws.col.info ~ "SDB"): 25 -> 500
• number of occurrences the packets were more than 9ms apart (frame.time_delta > 0.009): 6 -> 330
• note the above gaps happen after SDB
• note even the lower bound for time wasted in those gaps (330*0.009 = 3s) is 500% of original download time

These observations makes me to believe the flow control become too tight for certain link parameters.

I expected the following

Reference download takes approximately 0.6s.

curl/libcurl version

Custom build w/ the following component versions:
OpenSSL: openssl-3.6.0
nghttp3: v1.13.0-54-ga1a0551
ngtcp2: v1.18.0-77-g61ee993a
nghttp2: v1.68.0-72-gae9dedbc
curl: curl-8_17_0-277-g24b36fdd15

operating system

Ubuntu 22.04

[icing]

Thanks for testing this and reporting. I can indeed reproduce that my fine-tuning of ratelimiting crippled the download speed with ngtcp2. The initial stream window size was set small and never grown.

I improved the implementation of grow the stream window now in line with rate-limited and unlimited transfers in #20033. In my testing, this gives the old performance back.

Would be happy to hear how this fares in your testing.

[koujaz]

Re-tested the reference download times w/ following commits:
038ca9c (your PR) => 0.6s
3233304 (latest master; commit before your PR) => 5.8s

So yes. From my perspective the problem is fixed ;)
Thank you for prompt solution!