Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

scorecard: tweak request measurements #14564

Closed
wants to merge 1 commit into from

Conversation

icing
Copy link
Contributor

@icing icing commented Aug 15, 2024

Increase max-parallel up to 300, the curl max. Tweak output to just give the http response code.

This test is interesting in showing curl performance when having many, small requests. Changing the output from -w {%json}\n to -w %{response_code},\\n gives a 40% performance boost.

Some results from my macOS machine (with %{response_code}):

macOS
curl > CURL_TEST_EVENT=1 python3 tests/http/scorecard.py -r --request-count=100000 h2
H2 in curl 8.10.0-DEV (x86_64-apple-darwin23.6.0) libcurl/8.10.0-DEV quictls/3.1.4 zlib/1.2.12 brotli/1.1.0 zstd/1.5.6 c-ares/1.28.1 libidn2/2.3.7 libpsl/0.21.5 libssh2/1.11.0 nghttp2/1.62.1 ngtcp2/1.6.0 nghttp3/1.4.0 librtmp/2.3
Requests, max in parallel
  Server     Size   Reqs         1 [cpu/rss]             6 [cpu/rss]            25 [cpu/rss]            50 [cpu/rss]           100 [cpu/rss]           300 [cpu/rss]       Errors
libuv     httpd      10KB 100000  1869 r/s [85.4%/10MB]   3737 r/s [78.8%/12MB]   4348 r/s [83.5%/18MB]   4401 r/s [83.4%/25MB]   4322 r/s [83.6%/40MB]   4352 r/s [83.6%/62MB]      -
libuv     caddy      10KB 100000  1815 r/s [85.8%/10MB]   3491 r/s [83.4%/13MB]   3931 r/s [84.1%/21MB]   3946 r/s [84.2%/30MB]   3967 r/s [84.5%/45MB]   4014 r/s [84.9%/71MB]      -
mperform  httpd      10KB 100000  1785 r/s [65.8%/10MB]   3832 r/s [83.7%/12MB]   4334 r/s [83.5%/19MB]   4304 r/s [83.6%/26MB]   4332 r/s [83.8%/37MB]   4203 r/s [83.9%/55MB]      -
mperform  caddy      10KB 100000  1697 r/s [65.5%/10MB]   3320 r/s [84.2%/13MB]   3711 r/s [84.6%/21MB]   3793 r/s [84.8%/28MB]   3825 r/s [84.9%/42MB]   3882 r/s [85.2%/62MB]      -

debian sid
H2 in curl 8.10.0-DEV (x86_64-pc-linux-gnu) libcurl/8.10.0-DEV GnuTLS/3.7.7 zlib/1.3.1 brotli/1.1.0 zstd/1.5.6 c-ares/1.33.0 libidn2/2.3.7 libpsl/0.21.2 libssh2/1.11.0 nghttp2/1.61.0 ngtcp2/1.4.0 nghttp3/1.2.0 librtmp/2.3 OpenLDAP/2.5.18
Requests, max in parallel
Scheduler Server     Size   Reqs         1 [cpu/rss]             6 [cpu/rss]            25 [cpu/rss]            50 [cpu/rss]           100 [cpu/rss]           300 [cpu/rss]       Errors
libuv     httpd      10KB 100000  2902 r/s [100.0%/18MB]  3187 r/s [71.2%/18MB]   7406 r/s [100.0%/19MB]  7411 r/s [100.0%/19MB]  7424 r/s [100.0%/20MB]  6992 r/s [100.0%/33MB]     -
libuv     caddy      10KB 100000  2959 r/s [99.9%/18MB]   3502 r/s [68.7%/19MB]   6712 r/s [99.9%/19MB]   6869 r/s [100.0%/19MB]  6812 r/s [99.9%/20MB]   6606 r/s [100.0%/33MB]     -
mperform  httpd      10KB 100000  1090 r/s [61.0%/18MB]   6536 r/s [100.0%/18MB]  7263 r/s [99.9%/19MB]   7167 r/s [100.1%/19MB]  7091 r/s [100.0%/20MB]  6925 r/s [99.9%/28MB]      -
mperform  caddy      10KB 100000  2977 r/s [72.2%/18MB]   5562 r/s [99.8%/19MB]   6485 r/s [99.9%/19MB]   6744 r/s [100.0%/19MB]  6784 r/s [100.0%/19MB]  6529 r/s [100.0%/32MB]     -

On macOS, neither curl nor httpd are limited by their single cpu core (httpd load: 25% - 45%, caddy load: 45% - 95%
).

On linux the performance roughly doubles (although the cpu is slower) and differences between the libuv enabled scheduling and curl_multi_perform become visible. Curious about the delayed ramp-up of libuv at 6 parallel requests.

Update2: the reason why the 6 parallel requests on libuv do not scale up seems to be that the transfer loop in tool_operate.c does not add new handles often enough. If no timeout happens, the only opportunity to add transfers is when the last socket is removed and uv_run() returns.

Increase max-parallel up to 300, the curl max

Tweak output to just give the http response code
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

1 participant