Skip to content

tool: fix memory use in parallel mode - #22277

Closed
icing wants to merge 1 commit into
curl:masterfrom
icing:tool-parallel-mem-fix
Closed

tool: fix memory use in parallel mode#22277
icing wants to merge 1 commit into
curl:masterfrom
icing:tool-parallel-mem-fix

Conversation

@icing

@icing icing commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

The curl tool was creating a new transfer every time it checked if it needed to add one to reach max parallelism. This led to eventually all configured transfers to have easy handles created.

Limit the creation again to the ones needed for max parallelism.

scorecard.py: set --out-null only for curl versions that support it

When measuring mem use in scorecard for a non-debug curl build, notice the memory use when going from 1 parallel transfer to 6:

> python3 tests/http/scorecard.py -r --request-count=20000 --samples=3 h2
curl  Requests in parallel to httpd/2.4.67
version  size  total     1 max [cpu/rss]          6 max [cpu/rss]         25 max [cpu/rss]         50 max [cpu/rss]        100 max [cpu/rss]        300 max [cpu/rss]
8.15.0   10KB  20000  3130 r/s [50.3%/12MB]    9178 r/s [92.5%/13MB]   10827 r/s [98.3%/19MB]   10912 r/s [99.0%/26MB]   10857 r/s [99.4%/36MB]   10408 r/s [99.6%/65MB]
8.16.0   10KB  20000  3351 r/s [47.4%/12MB]   10851 r/s [94.9%/41MB]   14171 r/s [97.5%/31MB]   14657 r/s [98.8%/30MB]   15044 r/s [99.3%/27MB]   14802 r/s [99.5%/32MB]
8.17.0   10KB  20000  3391 r/s [47.2%/12MB]   10879 r/s [94.2%/42MB]   14501 r/s [98.1%/33MB]   15021 r/s [98.8%/30MB]   15337 r/s [99.3%/26MB]   15199 r/s [99.5%/31MB]
8.18.0   10KB  20000  3474 r/s [50.8%/11MB]   10394 r/s [95.6%/44MB]   13005 r/s [98.2%/34MB]   13451 r/s [98.8%/29MB]   13668 r/s [99.3%/27MB]   13701 r/s [99.6%/31MB]
8.19.0   10KB  20000  3430 r/s [50.3%/12MB]   10614 r/s [95.8%/39MB]   13099 r/s [96.2%/32MB]   13629 r/s [98.8%/28MB]   14066 r/s [99.3%/25MB]   13726 r/s [99.6%/28MB]
8.20.0   10KB  20000  3446 r/s [50.5%/11MB]   10768 r/s [95.4%/59MB]   13365 r/s [98.3%/42MB]   13749 r/s [98.8%/34MB]   14136 r/s [99.3%/30MB]   13946 r/s [99.6%/30MB]
8.21.0   10KB  20000  3395 r/s [50.9%/11MB]   10634 r/s [95.3%/57MB]   13370 r/s [98.3%/38MB]   13506 r/s [98.8%/33MB]   13908 r/s [99.3%/30MB]   13734 r/s [99.4%/30MB]
master   10KB  20000  3952 r/s [48.6%/11MB]   12184 r/s [95.1%/58MB]   15636 r/s [99.8%/44MB]   16070 r/s [99.8%/36MB]   15867 r/s [99.8%/29MB]   15841 r/s [99.9%/30MB]
PR       10KB  20000  3984 r/s [49.2%/11MB]   12711 r/s [95.1%/11MB]   16034 r/s [99.8%/12MB]   16596 r/s [99.9%/14MB]   16503 r/s [99.9%/16MB]   16303 r/s [99.9%/25MB]

The curl tool was creating a new transfer every time it checked if
it needed to add one to reach max parallelism. This led to eventually
all configured transfers to have easy handles created.

Limit the creation again to the ones needed for max parallelism.

scorecard.py: set --out-null only for curl versions that support it
@testclutch

Copy link
Copy Markdown

Analysis of PR #22277 at 4399487a:

Test 311 failed, but it has been 2.1% flaky lately, so it's probably NOT a fault of the PR. Note that this test has failed in 12 different CI jobs (the link just goes to one of them).

Generated by Testclutch

@icing
icing requested a review from bagder July 9, 2026 11:10
@bagder bagder closed this in eb73af2 Jul 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Development

Successfully merging this pull request may close these issues.

3 participants