Improve WTX by c410-f3r · Pull Request #599 · MDA2AV/HttpArena

c410-f3r · 2026-04-22T11:29:28Z

Description

Looks like the performance of WTX is terrible, which is strange given other similar benchmarks and the time I spent optimizing it.

Well, this is the last attempt without diving into profiling applications . Let's see if things will improve.

c410-f3r · 2026-04-22T11:31:34Z

/benchmark -f wtx-http2 --save

github-actions · 2026-04-22T11:31:42Z

👋 /benchmark request received. A collaborator will review and approve the run.

github-actions · 2026-04-22T11:31:58Z

🚀 Benchmark run triggered for wtx-http2 (all tests) with --save. Results will be posted here when done.

github-actions · 2026-04-22T11:39:00Z

Benchmark Results

Framework: wtx-http2 | Test: all tests

Test	Conn	RPS	CPU	Mem	Δ RPS	Δ Mem
baseline-h2c	256	660,468	4966.2%	665MiB	-6.1%	-4.3%
baseline-h2c	1024	662,474	5726.9%	1.5GiB	-2.7%	-6.3%
json-h2c	4096	605,907	4316.0%	1.9GiB	-2.4%	+5.6%

Full log

progress: Warm-up phase is over for thread #56.
% of clients started
40.
61.
3.
9.
Application protocol: h2c



58
. Stopping all clients.
52. Stopping all clients.


. Stopping all clients.
51

finished in 5.06s, 670424.40 req/s, 23.03MB/s
requests: 3352122 total, 3454522 started, 3352122 done, 3352122 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 3352122 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 115.15MB (120744129) total, 54.35MB (56986074) headers (space savings 58.54%), 3.20MB (3352122) data
                     min         max         mean         sd        +/- sd
time for request:       36us       1.04s    122.05ms    117.61ms    79.69%
time for connect:       19us       2.08s    429.31ms    537.89ms    59.96%
time to 1st byte:     1.51ms       2.28s    486.12ms    599.28ms    60.73%
req/s           :       0.00     1139.90      654.62      183.28    67.19%
[info] CPU 5726.9% | Mem 1.5GiB

[run 3/3]
starting benchmark...
progress: Main benchmark duration is started for thread #36.
.Main benchmark duration is started for thread #Main benchmark duration is started for thread #54Main benchmark duration is started for thread #22.
.
.

.


5.
81Main benchmark duration is started for thread #52.
% of clients started
.

8Main benchmark duration is started for thread #10.
.Main benchmark duration is started for thread #23.

.
Application protocol: h2c
21.

8. Stopping all clients.

23. Stopping all clients.
57. Stopping all clients.
45. Stopping all clients.

finished in 5.05s, 665371.80 req/s, 22.86MB/s
requests: 3326859 total, 3429259 started, 3326859 done, 3326859 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 3326859 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 114.28MB (119833254) total, 53.94MB (56556603) headers (space savings 58.54%), 3.17MB (3326859) data
                     min         max         mean         sd        +/- sd
time for request:       38us       1.27s    124.96ms    117.90ms    75.70%
time for connect:       19us       1.05s    325.55ms    477.87ms    68.36%
time to 1st byte:     2.09ms       1.29s    354.64ms    525.88ms    70.71%
req/s           :       0.00     1040.21      649.69      181.63    76.86%
[info] CPU 5729.9% | Mem 1.6GiB

=== Best: 662474 req/s (CPU: 5726.9%, Mem: 1.5GiB) ===
[info] saved results/baseline-h2c/1024/wtx-http2.json
httparena-bench-wtx-http2
httparena-bench-wtx-http2

==============================================
=== wtx-http2 / json-h2c / 4096c (tool=h2load) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
starting benchmark...

Application protocol: h2c
26.
.
.
36.
.
44.
10Main benchmark duration is started for thread #44.
.



41. Stopping all clients.
58. Stopping all clients.

finished in 5.11s, 604092.60 req/s, 2.09GB/s
requests: 3020463 total, 3151535 started, 3020463 done, 3020463 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 3020463 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 10.46GB (11227954502) total, 60.49MB (63429723) headers (space savings 55.32%), 10.35GB (11109882080) data
                     min         max         mean         sd        +/- sd
time for request:       50us       3.86s    137.41ms    248.61ms    92.80%
time for connect:       14us       3.04s    853.92ms    675.08ms    58.28%
time to 1st byte:     1.50ms       4.89s       1.63s       1.31s    60.83%
req/s           :       0.00      412.66      147.42       87.77    65.58%
[info] CPU 4171.1% | Mem 1.8GiB

[run 2/3]
starting benchmark...
.16
.Main benchmark duration is started for thread #61.


20Warm-up phase is over for thread #Warm-up phase is over for thread #45.
23.
3145.
.Main benchmark duration is started for thread #31.
.
.Main benchmark duration is started for thread #15.

3.
44.


Application protocol: h2c
41
35. Stopping all clients.

. Stopping all clients.Stopped all clients for thread #Main benchmark duration is over for thread #43
40

59
25

finished in 5.09s, 616814.20 req/s, 2.14GB/s
requests: 3084071 total, 3215143 started, 3084071 done, 3084071 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 3084071 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 10.68GB (11465031138) total, 61.77MB (64765491) headers (space savings 55.32%), 10.57GB (11344477937) data
                     min         max         mean         sd        +/- sd
time for request:       46us       3.84s    141.35ms    214.73ms    86.32%
time for connect:       15us       2.08s    772.42ms    706.89ms    46.17%
time to 1st byte:     1.24ms       4.87s       1.24s       1.11s    60.67%
req/s           :       0.20      393.87      150.52       73.87    63.50%
[info] CPU 4316.0% | Mem 1.9GiB

[run 3/3]
starting benchmark...

.Main benchmark duration is started for thread #2
.
42Main benchmark duration is started for thread #53.
.
21.
Application protocol: h2c
13.
4.
57.
.
19Stopped all clients for thread #22
. Stopping all clients.
20. Stopping all clients.
44. Stopping all clients.
6


finished in 5.08s, 612046.20 req/s, 2.12GB/s
requests: 3060231 total, 3191303 started, 3060231 done, 3060231 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 3060231 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 10.59GB (11375675278) total, 61.29MB (64264851) headers (space savings 55.32%), 10.48GB (11256051837) data
                     min         max         mean         sd        +/- sd
time for request:       52us       1.60s    136.60ms    174.75ms    83.73%
time for connect:       16us       4.11s    943.69ms    924.60ms    38.94%
time to 1st byte:     1.55ms       4.48s       1.23s       1.08s    47.36%
req/s           :      17.80      370.75      149.37       72.01    61.77%
[info] CPU 4267.9% | Mem 2.0GiB

=== Best: 605907 req/s (CPU: 4316.0%, Mem: 1.9GiB) ===
[info] saved results/json-h2c/4096/wtx-http2.json
httparena-bench-wtx-http2
httparena-bench-wtx-http2
[info] skip: wtx-http2 does not subscribe to baseline-h3
[info] skip: wtx-http2 does not subscribe to static-h3
[info] skip: wtx-http2 does not subscribe to gateway-64
[info] skip: wtx-http2 does not subscribe to gateway-h3
[info] skip: wtx-http2 does not subscribe to production-stack
[info] skip: wtx-http2 does not subscribe to unary-grpc
[info] skip: wtx-http2 does not subscribe to unary-grpc-tls
[info] skip: wtx-http2 does not subscribe to stream-grpc
[info] skip: wtx-http2 does not subscribe to stream-grpc-tls
[info] skip: wtx-http2 does not subscribe to echo-ws
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2c-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2c-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-h2c-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
[info] restoring loopback MTU to 65536
[info] restoring CPU governor → powersave

MDA2AV · 2026-04-22T11:42:15Z

I'll add some more frameworks on this test for a broader comparison, could be a load generator issue or not optimal config on connections/streams

c410-f3r · 2026-04-22T11:54:52Z

Hum, seems unlikely given the results of the other HTTP/2 frameworks.

It is probably a bad default HTTP/2 configuration like the maximum number of concurrent streams or a recent change I made in the WTX project.

c410-f3r · 2026-04-22T13:44:31Z

Hum, let me tweak some HTTP/2 parameters to see their runtime impact. I will be back after a couple of hours.

MDA2AV · 2026-04-22T15:07:02Z

Added a few more h2c entries and more conn variety for better comparison

https://www.http-arena.com/leaderboard/#v=h2&t=baseline-h2c&type=infrastructure,production,tuned

c410-f3r · 2026-04-22T18:59:36Z

Added a few more h2c entries and more conn variety for better comparison

https://www.http-arena.com/leaderboard/#v=h2&t=baseline-h2c&type=infrastructure,production,tuned

Wow... The performance of h2o is really impressive

Not so much in the JSON scenario, strange.

c410-f3r · 2026-04-22T18:59:48Z

/benchmark -f wtx-http2 --save

github-actions · 2026-04-22T18:59:55Z

👋 /benchmark request received. A collaborator will review and approve the run.

github-actions · 2026-04-22T19:00:13Z

🚀 Benchmark run triggered for wtx-http2 (all tests) with --save. Results will be posted here when done.

github-actions · 2026-04-22T19:13:17Z

Benchmark Results

Framework: wtx-http2 | Test: all tests

Test	Conn	RPS	CPU	Mem	Δ RPS	Δ Mem
baseline-h2c	256	658,273	4554.2%	605MiB	-3.4%	-7.9%
baseline-h2c	1024	663,592	5459.2%	1.3GiB	+1.9%	-7.1%
baseline-h2c	4096	638,297	4324.0%	2.4GiB	-5.5%	-14.3%
json-h2c	1024	640,863	4356.7%	614MiB	+6.9%	-19.8%
json-h2c	4096	606,637	5294.6%	1.3GiB	+5.3%	-27.8%

Full log

.
8.


43. Stopping all clients.
51. Stopping all clients.
16
. Stopping all clients.Stopped all clients for thread #31


37. Stopping all clients.

finished in 5.06s, 643712.20 req/s, 2.23GB/s
requests: 3218561 total, 3251329 started, 3218561 done, 3218561 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 3218561 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 11.16GB (11983756789) total, 64.46MB (67589781) headers (space savings 55.32%), 11.04GB (11858164302) data
                     min         max         mean         sd        +/- sd
time for request:       43us    389.38ms     43.82ms     40.18ms    76.70%
time for connect:       18us       2.03s    338.28ms    574.90ms    71.48%
time to 1st byte:      920us       2.10s    389.69ms    620.40ms    71.48%
req/s           :     288.59      883.86      628.57      120.74    65.82%
[info] CPU 4379.7% | Mem 634MiB

=== Best: 640863 req/s (CPU: 4356.7%, Mem: 614MiB) ===
[info] saved results/json-h2c/1024/wtx-http2.json
httparena-bench-wtx-http2
httparena-bench-wtx-http2

==============================================
=== wtx-http2 / json-h2c / 4096c (tool=h2load) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
starting benchmark...
38.Main benchmark duration is started for thread #7.

.Main benchmark duration is started for thread #35.

60.Warm-up phase is over for thread #
40.
9.
31Main benchmark duration is started for thread #57.
.
Application protocol: h2c
.


.
11.
26Warm-up phase is over for thread #.
27.
7. Stopping all clients.
60
. Stopping all clients.
57. Stopping all clients.
. Stopping all clients.

. Stopping all clients.

20
52
4

finished in 5.10s, 609015.60 req/s, 2.11GB/s
requests: 3045078 total, 3176150 started, 3045078 done, 3045078 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 3045078 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 10.55GB (11323738831) total, 60.98MB (63946638) headers (space savings 55.32%), 10.44GB (11204749706) data
                     min         max         mean         sd        +/- sd
time for request:       46us       3.98s    114.11ms    222.80ms    94.14%
time for connect:       14us       3.06s       1.10s    843.24ms    47.90%
time to 1st byte:      919us       4.66s       1.80s       1.37s    55.78%
req/s           :       0.00      483.90      148.63      122.69    56.96%
[info] CPU 4999.0% | Mem 1.2GiB

[run 2/3]
starting benchmark...

0.
Application protocol: h2c
.
31.
.
.Main benchmark duration is started for thread #
10.
56.
.

.Warm-up phase is over for thread #Main benchmark duration is started for thread #51.1
.
1.

.
.Warm-up phase is over for thread #Main benchmark duration is started for thread #857..


55. Stopping all clients.Main benchmark duration is over for thread #
46. Stopping all clients.
54. Stopping all clients.
12. Stopping all clients.
9
51. Stopping all clients.
13
17


finished in 5.09s, 617557.20 req/s, 2.14GB/s
requests: 3087786 total, 3218858 started, 3087786 done, 3087786 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 3087786 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 10.69GB (11479519897) total, 61.84MB (64843506) headers (space savings 55.32%), 10.58GB (11358826166) data
                     min         max         mean         sd        +/- sd
time for request:       53us       3.73s    145.44ms    223.66ms    89.03%
time for connect:       17us       2.09s    601.77ms    528.20ms    56.45%
time to 1st byte:     1.84ms       4.77s       1.16s       1.19s    88.09%
req/s           :       0.00      337.00      150.69       71.40    65.70%
[info] CPU 5294.6% | Mem 1.3GiB

[run 3/3]
starting benchmark...
Application protocol: h2c
.
.Main benchmark duration is started for thread #44.

.
.
.Warm-up phase is over for thread #
58Main benchmark duration is started for thread #.52
.

.Warm-up phase is over for thread #Warm-up phase is over for thread #
408.Main benchmark duration is started for thread #48.

62Warm-up phase is over for thread #.
21.

58.
.
57.


32
10. Stopping all clients.Main benchmark duration is over for thread #0. Stopping all clients.

. Stopping all clients.
10
11Stopped all clients for thread #13
. Stopping all clients.
. Stopping all clients.

29Main benchmark duration is over for thread #. Stopping all clients.25. Stopping all clients.

12. Stopping all clients.


4Main benchmark duration is over for thread #28. Stopping all clients.
42Stopped all clients for thread #56
38. Stopping all clients.Stopped all clients for thread #Stopped all clients for thread #28

48Main benchmark duration is over for thread #
61. Stopping all clients.



finished in 5.10s, 615101.00 req/s, 2.13GB/s
requests: 3075505 total, 3206577 started, 3075505 done, 3075505 succeeded, 0 failed, 0 errored, 0 timeout
status codes: 3075505 2xx, 0 3xx, 0 4xx, 0 5xx
traffic: 10.65GB (11432588910) total, 61.59MB (64585605) headers (space savings 55.32%), 10.54GB (11312369783) data
                     min         max         mean         sd        +/- sd
time for request:       46us       3.73s    146.57ms    222.52ms    86.42%
time for connect:       19us       2.08s    631.65ms    575.79ms    53.76%
time to 1st byte:     1.63ms       4.77s       1.13s       1.05s    56.40%
req/s           :       0.20      353.34      150.10       67.44    63.06%
[info] CPU 5146.0% | Mem 1.4GiB

=== Best: 606637 req/s (CPU: 5294.6%, Mem: 1.3GiB) ===
[info] saved results/json-h2c/4096/wtx-http2.json
httparena-bench-wtx-http2
httparena-bench-wtx-http2
[info] skip: wtx-http2 does not subscribe to baseline-h3
[info] skip: wtx-http2 does not subscribe to static-h3
[info] skip: wtx-http2 does not subscribe to gateway-64
[info] skip: wtx-http2 does not subscribe to gateway-h3
[info] skip: wtx-http2 does not subscribe to production-stack
[info] skip: wtx-http2 does not subscribe to unary-grpc
[info] skip: wtx-http2 does not subscribe to unary-grpc-tls
[info] skip: wtx-http2 does not subscribe to stream-grpc
[info] skip: wtx-http2 does not subscribe to stream-grpc-tls
[info] skip: wtx-http2 does not subscribe to echo-ws
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2c-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2c-256.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/baseline-h2c-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-h2c-1024.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/json-h2c-4096.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
[info] restoring loopback MTU to 65536
[info] restoring CPU governor → performance

c410-f3r · 2026-04-22T19:17:34Z

Nah, I give up. That was the best I could do given the constraints.

At least it is clear that HTTP/2 parameters weren't the problem.

c410-f3r · 2026-04-22T19:19:37Z

Perhaps I will get back after a couple of months. The code changes are still worth merging though.

MDA2AV · 2026-04-22T19:25:09Z

Not so much in the JSON scenario, strange.

Yea, the json implementation is really bad for it, jsut threw it in

c410-f3r requested review from Kaliumhexacyanoferrat and MDA2AV as code owners April 22, 2026 11:29

Improve WTX

64013bf

c410-f3r force-pushed the main branch from d9b69fa to 64013bf Compare April 22, 2026 18:57

Benchmark results: wtx-http2

0b84567

Merge branch 'main' into main

d35f66a

MDA2AV approved these changes Apr 23, 2026

View reviewed changes

MDA2AV merged commit 20c2fa4 into MDA2AV:main Apr 23, 2026
4 checks passed

Conversation

c410-f3r commented Apr 22, 2026

Description

Uh oh!

c410-f3r commented Apr 22, 2026

Uh oh!

github-actions Bot commented Apr 22, 2026

Uh oh!

github-actions Bot commented Apr 22, 2026

Uh oh!

github-actions Bot commented Apr 22, 2026

Benchmark Results

Uh oh!

MDA2AV commented Apr 22, 2026

Uh oh!

c410-f3r commented Apr 22, 2026

Uh oh!

c410-f3r commented Apr 22, 2026

Uh oh!

MDA2AV commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

c410-f3r commented Apr 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

c410-f3r commented Apr 22, 2026

Uh oh!

github-actions Bot commented Apr 22, 2026

Uh oh!

github-actions Bot commented Apr 22, 2026

Uh oh!

github-actions Bot commented Apr 22, 2026

Benchmark Results

Uh oh!

c410-f3r commented Apr 22, 2026

Uh oh!

c410-f3r commented Apr 22, 2026

Uh oh!

MDA2AV commented Apr 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

MDA2AV commented Apr 22, 2026 •

edited

Loading

c410-f3r commented Apr 22, 2026 •

edited

Loading