Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use jemalloc in benchmarks #1849

Merged
merged 3 commits into from
Mar 14, 2024
Merged

Use jemalloc in benchmarks #1849

merged 3 commits into from
Mar 14, 2024

Conversation

ctz
Copy link
Member

@ctz ctz commented Mar 14, 2024 •
edited
Loading

This normalises the manual modifications from https://github.com/aochagavia/rustls-bench-results

Third commit adds aggressive LTO to the existing bench profile. Full results (--release vs --profile=bench, run on benchmark box):

$ diff-bench rustls.jemalloc.out.txt rustls.jemalloc-lto.out.txt 
bulk	TLSv1_2	TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256	max_fragment_size:default	send	7150.59	7070.55	MB/s	-1.1%
bulk	TLSv1_2	TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256	max_fragment_size:default	recv	6284.95	7018.19	MB/s	+12%
bulk	TLSv1_2	TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384	max_fragment_size:default	send	6729.37	6583.54	MB/s	-2.2%
bulk	TLSv1_2	TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384	max_fragment_size:default	recv	5942.65	6596.05	MB/s	+11%
bulk	TLSv1_2	TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256	max_fragment_size:default	recv	1753.77	1805.13	MB/s	+2.9%
bulk	TLSv1_3	TLS13_AES_256_GCM_SHA384	max_fragment_size:default	recv	6242.48	6971.74	MB/s	+12%
handshakes	TLSv1_2	Rsa	TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384	client	server-auth	no-resume	4997.66	5059.81	handshake/s	+1.2%
handshakes	TLSv1_2	Rsa	TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384	client	server-auth	sessionid	50515.9	52052.9	handshake/s	+3%
handshakes	TLSv1_2	Rsa	TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384	server	server-auth	sessionid	42204.1	43706.3	handshake/s	+3.6%
handshakes	TLSv1_2	Rsa	TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384	client	server-auth	tickets	47600.1	50216.2	handshake/s	+5.5%
handshakes	TLSv1_2	Rsa	TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384	server	server-auth	tickets	36821	38705.8	handshake/s	+5.1%
handshakes	TLSv1_3	Rsa	TLS13_AES_256_GCM_SHA384	client	server-auth	no-resume	4697.7	4793.21	handshake/s	+2%
handshakes	TLSv1_3	Rsa	TLS13_AES_256_GCM_SHA384	client	server-auth	sessionid	11383.6	11592.3	handshake/s	+1.8%
handshakes	TLSv1_3	Rsa	TLS13_AES_256_GCM_SHA384	server	server-auth	sessionid	7452.58	7629.46	handshake/s	+2.4%
handshakes	TLSv1_3	Rsa	TLS13_AES_256_GCM_SHA384	client	server-auth	tickets	11240	11630.8	handshake/s	+3.5%
handshakes	TLSv1_3	Rsa	TLS13_AES_256_GCM_SHA384	server	server-auth	tickets	7406.77	7583.08	handshake/s	+2.4%

fixes #1722

@rustls-benchmarking Rustls Benchmarking
Copy link

rustls-benchmarking bot commented Mar 14, 2024 •
edited
Loading

Benchmark results

Instruction counts

Significant differences

⚠️ There are significant instruction count differences

Click to expand
Scenario Baseline Candidate Diff Threshold
handshake_session_id_aws_lc_rs_1.2_rsa_aes_client 4179354 3970778 ✅ -208576 (-4.99%) 0.70%
handshake_tickets_aws_lc_rs_1.2_rsa_aes_client 4510763 4289174 ✅ -221589 (-4.91%) 0.75%
handshake_session_id_aws_lc_rs_1.2_rsa_aes_server 4055148 3877986 ✅ -177162 (-4.37%) 3.67%
handshake_session_id_ring_1.2_rsa_aes_client 4434319 4245574 ✅ -188745 (-4.26%) 0.59%
handshake_tickets_ring_1.2_rsa_aes_client 4709851 4513893 ✅ -195958 (-4.16%) 0.69%
handshake_tickets_aws_lc_rs_1.2_rsa_aes_server 4539104 4357750 ✅ -181354 (-4.00%) 3.21%
handshake_tickets_ring_1.2_rsa_aes_server 4767865 4665862 ✅ -102003 (-2.14%) 0.28%
handshake_session_id_ring_1.2_rsa_aes_server 4328541 4240999 ✅ -87542 (-2.02%) 0.32%
handshake_tickets_aws_lc_rs_1.3_rsa_aes_client 30659687 30344614 ✅ -315073 (-1.03%) 0.20%
handshake_session_id_aws_lc_rs_1.3_rsa_aes_client 30459249 30146427 ✅ -312822 (-1.03%) 0.20%
handshake_tickets_aws_lc_rs_1.3_rsa_chacha_client 30636053 30332708 ✅ -303345 (-0.99%) 0.20%
handshake_session_id_aws_lc_rs_1.3_rsa_chacha_client 30429158 30131140 ✅ -298018 (-0.98%) 0.20%
handshake_session_id_ring_1.3_rsa_chacha_client 42119124 41813851 ✅ -305273 (-0.72%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_client 3344437 3368575 ⚠️ 24138 (0.72%) 0.24%
handshake_no_resume_aws_lc_rs_1.2_rsa_aes_client 3151798 3174426 ⚠️ 22628 (0.72%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_rsa_aes_client 3356188 3379564 ⚠️ 23376 (0.70%) 0.20%
handshake_session_id_ring_1.3_rsa_aes_client 42187825 41894231 ✅ -293594 (-0.70%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_rsa_chacha_client 3366600 3389937 ⚠️ 23337 (0.69%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_aes_client 3341086 3363241 ⚠️ 22155 (0.66%) 0.20%
handshake_tickets_ring_1.3_rsa_chacha_client 42292013 42019103 ✅ -272910 (-0.65%) 0.20%
handshake_no_resume_ring_1.3_ecdsap256_chacha_client 3895135 3919913 ⚠️ 24778 (0.64%) 0.24%
handshake_tickets_ring_1.3_rsa_aes_client 42357251 42088356 ✅ -268895 (-0.63%) 0.20%
handshake_no_resume_ring_1.3_ecdsap256_aes_client 3896284 3920599 ⚠️ 24315 (0.62%) 0.22%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_aes_client 30284181 30101164 ✅ -183017 (-0.60%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_chacha_client 30234941 30072931 ✅ -162010 (-0.54%) 0.40%
handshake_no_resume_ring_1.3_ecdsap256_aes_server 2124803 2136071 ⚠️ 11268 (0.53%) 0.38%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_aes_server 1879643 1889194 ⚠️ 9551 (0.51%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_chacha_client 30236161 30084699 ✅ -151462 (-0.50%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_chacha_server 32466967 32305294 ✅ -161673 (-0.50%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_server 1883046 1892394 ⚠️ 9348 (0.50%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_aes_client 30260625 30111092 ✅ -149533 (-0.49%) 0.44%
transfer_no_resume_ring_1.2_rsa_aes_client 57919950 58202168 ⚠️ 282218 (0.49%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_chacha_client 30406522 30259174 ✅ -147348 (-0.48%) 0.29%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_aes_client 30439210 30301382 ✅ -137828 (-0.45%) 0.20%
handshake_no_resume_ring_1.3_ecdsap256_chacha_server 2126338 2135925 ⚠️ 9587 (0.45%) 0.35%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_aes_server 32478649 32333221 ✅ -145428 (-0.45%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_aes_client 30451457 30315159 ✅ -136298 (-0.45%) 0.40%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_aes_client 57963195 58221463 ⚠️ 258268 (0.45%) 0.20%
transfer_no_resume_ring_1.3_rsa_aes_client 58063782 58320168 ⚠️ 256386 (0.44%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_aes_client 58063003 58316565 ⚠️ 253562 (0.44%) 0.20%
transfer_no_resume_ring_1.3_ecdsap384_aes_client 58063679 58315946 ⚠️ 252267 (0.43%) 0.20%
handshake_no_resume_ring_1.2_rsa_aes_client 4440538 4459368 ⚠️ 18830 (0.42%) 0.20%
handshake_no_resume_ring_1.3_rsa_chacha_client 4547663 4566723 ⚠️ 19060 (0.42%) 0.20%
handshake_no_resume_ring_1.3_rsa_aes_client 4538046 4556925 ⚠️ 18879 (0.42%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_aes_client 57985034 58221249 ⚠️ 236215 (0.41%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_aes_client 57996860 58231025 ⚠️ 234165 (0.40%) 0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_chacha_server 32435231 32304560 ✅ -130671 (-0.40%) 0.25%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_aes_server 32463753 32333506 ✅ -130247 (-0.40%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_chacha_client 30407470 30286116 ✅ -121354 (-0.40%) 0.20%
transfer_no_resume_aws_lc_rs_1.2_rsa_aes_client 68385656 68641583 ⚠️ 255927 (0.37%) 0.20%
handshake_session_id_ring_1.3_ecdsap384_chacha_client 41915610 41761342 ✅ -154268 (-0.37%) 0.20%
transfer_no_resume_aws_lc_rs_1.2_rsa_aes_server 46212822 46380695 ⚠️ 167873 (0.36%) 0.29%
transfer_no_resume_ring_1.3_ecdsap384_aes_server 46289758 46457831 ⚠️ 168073 (0.36%) 0.20%
transfer_no_resume_ring_1.3_rsa_aes_server 46289810 46457864 ⚠️ 168054 (0.36%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_aes_server 46254454 46419743 ⚠️ 165289 (0.36%) 0.20%
transfer_no_resume_ring_1.2_rsa_aes_server 46175646 46338600 ⚠️ 162954 (0.35%) 0.20%
handshake_session_id_ring_1.3_ecdsap256_aes_server 43563287 43411211 ✅ -152076 (-0.35%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_aes_server 46293093 46452605 ⚠️ 159512 (0.34%) 0.20%
handshake_session_id_ring_1.3_ecdsap384_aes_client 41986215 41842022 ✅ -144193 (-0.34%) 0.20%
handshake_session_id_ring_1.3_ecdsap256_chacha_client 41907127 41764354 ✅ -142773 (-0.34%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_aes_server 46264944 46418480 ⚠️ 153536 (0.33%) 0.20%
handshake_session_id_ring_1.3_rsa_aes_server 43550640 43408944 ✅ -141696 (-0.33%) 0.20%
handshake_session_id_ring_1.3_ecdsap256_aes_client 41977542 41841046 ✅ -136496 (-0.33%) 0.20%
handshake_session_id_ring_1.3_rsa_chacha_server 43450047 43311046 ✅ -139001 (-0.32%) 0.20%
handshake_session_id_ring_1.3_ecdsap384_chacha_server 43448420 43311012 ✅ -137408 (-0.32%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_aes_client 42161460 42029137 ✅ -132323 (-0.31%) 0.20%
handshake_session_id_ring_1.3_ecdsap256_chacha_server 43452548 43317913 ✅ -134635 (-0.31%) 0.20%
handshake_session_id_ring_1.3_ecdsap384_aes_server 43543436 43408805 ✅ -134631 (-0.31%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_chacha_client 42089538 41962371 ✅ -127167 (-0.30%) 0.20%
handshake_tickets_ring_1.3_ecdsap384_aes_client 42148803 42030622 ✅ -118181 (-0.28%) 0.20%
transfer_no_resume_ring_1.3_rsa_chacha_client 92397495 92652283 ⚠️ 254788 (0.28%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_chacha_client 92400509 92653778 ⚠️ 253269 (0.27%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_client 92434084 92687288 ⚠️ 253204 (0.27%) 0.20%
transfer_no_resume_ring_1.3_ecdsap384_chacha_client 92392166 92644686 ⚠️ 252520 (0.27%) 0.20%
handshake_tickets_ring_1.3_ecdsap384_chacha_client 42077606 41966599 ✅ -111007 (-0.26%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_chacha_client 92455508 92689323 ⚠️ 233815 (0.25%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_client 92428470 92646078 ⚠️ 217608 (0.24%) 0.20%
handshake_tickets_ring_1.3_rsa_aes_server 43791440 43690831 ✅ -100609 (-0.23%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_server 80436452 80610577 ⚠️ 174125 (0.22%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_aes_server 4258219 4267215 ⚠️ 8996 (0.21%) 0.20%
handshake_tickets_ring_1.3_rsa_chacha_server 43710021 43618147 ✅ -91874 (-0.21%) 0.21%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_server 4262124 4271082 ⚠️ 8958 (0.21%) 0.20%
transfer_no_resume_ring_1.3_rsa_chacha_server 80342695 80510594 ⚠️ 167899 (0.21%) 0.20%
transfer_no_resume_ring_1.3_ecdsap256_chacha_server 80348063 80509503 ⚠️ 161440 (0.20%) 0.20%

Other differences

Click to expand
Scenario Baseline Candidate Diff Threshold
handshake_no_resume_aws_lc_rs_1.2_rsa_aes_server 12285046 12347471 62425 (0.51%) 0.95%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_client 8671727 8709676 37949 (0.44%) 0.66%
handshake_no_resume_aws_lc_rs_1.3_rsa_chacha_server 12689633 12740259 50626 (0.40%) 0.80%
handshake_tickets_aws_lc_rs_1.3_rsa_aes_server 32693809 32570634 -123175 (-0.38%) 0.43%
handshake_session_id_aws_lc_rs_1.3_rsa_aes_server 32465245 32351427 -113818 (-0.35%) 0.47%
handshake_session_id_aws_lc_rs_1.3_rsa_chacha_server 32418381 32306314 -112067 (-0.35%) 0.46%
handshake_tickets_aws_lc_rs_1.3_rsa_chacha_server 32666192 32558593 -107599 (-0.33%) 0.71%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_aes_client 8652866 8681264 28398 (0.33%) 0.83%
transfer_no_resume_aws_lc_rs_1.3_rsa_aes_server 46288643 46415505 126862 (0.27%) 0.36%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_chacha_server 32657566 32579832 -77734 (-0.24%) 0.25%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_aes_server 32655726 32591344 -64382 (-0.20%) 0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_aes_server 32654758 32590849 -63909 (-0.20%) 0.20%
transfer_no_resume_ring_1.3_ecdsap384_chacha_server 80350087 80507066 156979 (0.20%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_server 80455473 80608852 153379 (0.19%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_chacha_server 43677814 43621686 -56128 (-0.13%) 0.20%
handshake_no_resume_aws_lc_rs_1.3_rsa_aes_server 12675713 12691631 15918 (0.13%) 0.87%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_chacha_server 32617306 32578983 -38323 (-0.12%) 0.24%
handshake_tickets_ring_1.3_ecdsap384_chacha_server 43667891 43621363 -46528 (-0.11%) 0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_chacha_server 80471884 80555813 83929 (0.10%) 0.25%
handshake_tickets_ring_1.3_ecdsap384_aes_server 43725487 43688366 -37121 (-0.08%) 0.20%
handshake_tickets_ring_1.3_ecdsap256_aes_server 43726382 43691864 -34518 (-0.08%) 0.20%
handshake_no_resume_ring_1.3_ecdsap384_aes_server 13734620 13743977 9357 (0.07%) 0.20%
handshake_no_resume_ring_1.3_ecdsap384_aes_client 35452389 35475475 23086 (0.07%) 0.20%
handshake_no_resume_ring_1.3_ecdsap384_chacha_server 13737871 13746751 8880 (0.06%) 0.20%
handshake_no_resume_ring_1.3_ecdsap384_chacha_client 35455167 35477637 22470 (0.06%) 0.20%
handshake_no_resume_ring_1.2_rsa_aes_server 12043160 12049453 6293 (0.05%) 0.20%
handshake_no_resume_ring_1.3_rsa_chacha_server 12247759 12252757 4998 (0.04%) 0.20%
handshake_no_resume_ring_1.3_rsa_aes_server 12238205 12243189 4984 (0.04%) 0.20%

Wall-time

Significant differences

⚠️ There are significant wall-time differences

Click to expand
Scenario Baseline Candidate Diff Threshold
handshake_tickets_ring_1.2_rsa_aes 1.85 ms 1.74 ms ✅ -0.11 ms (-6.20%) 1.59%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_aes 4.73 ms 4.44 ms ✅ -0.29 ms (-6.19%) 4.53%
handshake_session_id_ring_1.2_rsa_aes 1.76 ms 1.66 ms ✅ -0.10 ms (-5.52%) 2.14%
handshake_session_id_aws_lc_rs_1.2_rsa_aes 2.18 ms 2.06 ms ✅ -0.12 ms (-5.51%) 1.78%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_aes 5.46 ms 5.16 ms ✅ -0.30 ms (-5.50%) 3.74%
handshake_tickets_aws_lc_rs_1.2_rsa_aes 2.35 ms 2.23 ms ✅ -0.12 ms (-5.17%) 1.71%
transfer_no_resume_aws_lc_rs_1.2_rsa_aes 5.67 ms 5.38 ms ✅ -0.29 ms (-5.08%) 3.58%
transfer_no_resume_aws_lc_rs_1.3_rsa_aes 5.68 ms 5.40 ms ✅ -0.29 ms (-5.06%) 3.52%
transfer_no_resume_ring_1.3_ecdsap256_aes 6.58 ms 6.28 ms ✅ -0.29 ms (-4.44%) 3.28%
transfer_no_resume_ring_1.2_rsa_aes 7.07 ms 6.79 ms ✅ -0.29 ms (-4.03%) 2.87%
transfer_no_resume_ring_1.3_rsa_aes 7.15 ms 6.88 ms ✅ -0.27 ms (-3.74%) 2.67%
transfer_no_resume_ring_1.3_ecdsap384_aes 9.68 ms 9.39 ms ✅ -0.29 ms (-2.96%) 2.61%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_aes 5.39 ms 5.24 ms ✅ -0.15 ms (-2.82%) 1.42%
handshake_tickets_aws_lc_rs_1.3_rsa_aes 6.38 ms 6.20 ms ✅ -0.17 ms (-2.74%) 1.19%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_chacha 5.39 ms 5.25 ms ✅ -0.14 ms (-2.58%) 1.51%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_aes 5.36 ms 5.22 ms ✅ -0.14 ms (-2.52%) 1.45%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_aes 6.10 ms 5.95 ms ✅ -0.15 ms (-2.46%) 1.37%
handshake_session_id_aws_lc_rs_1.3_rsa_aes 6.35 ms 6.20 ms ✅ -0.16 ms (-2.46%) 1.21%
handshake_tickets_aws_lc_rs_1.3_rsa_chacha 6.37 ms 6.22 ms ✅ -0.16 ms (-2.45%) 1.53%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_chacha 13.23 ms 12.92 ms ✅ -0.31 ms (-2.31%) 1.41%
transfer_no_resume_ring_1.3_ecdsap256_chacha 13.23 ms 12.93 ms ✅ -0.30 ms (-2.29%) 1.56%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_chacha 6.09 ms 5.95 ms ✅ -0.14 ms (-2.27%) 1.62%
handshake_session_id_aws_lc_rs_1.3_rsa_chacha 6.34 ms 6.20 ms ✅ -0.14 ms (-2.21%) 1.42%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_chacha 5.34 ms 5.22 ms ✅ -0.12 ms (-2.17%) 1.82%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_aes 6.07 ms 5.94 ms ✅ -0.13 ms (-2.17%) 1.35%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_chacha 13.93 ms 13.63 ms ✅ -0.30 ms (-2.13%) 1.40%
transfer_no_resume_aws_lc_rs_1.3_rsa_chacha 14.17 ms 13.87 ms ✅ -0.30 ms (-2.13%) 1.37%
transfer_no_resume_ring_1.3_rsa_chacha 13.81 ms 13.52 ms ✅ -0.29 ms (-2.07%) 1.50%
handshake_session_id_ring_1.3_rsa_chacha 7.47 ms 7.31 ms ✅ -0.15 ms (-2.05%) 1.15%
handshake_session_id_ring_1.3_rsa_aes 7.51 ms 7.36 ms ✅ -0.15 ms (-2.05%) 1.00%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_chacha 6.04 ms 5.92 ms ✅ -0.12 ms (-1.97%) 1.59%
handshake_tickets_ring_1.3_rsa_aes 7.53 ms 7.39 ms ✅ -0.15 ms (-1.95%) 1.02%
handshake_tickets_ring_1.3_rsa_chacha 7.49 ms 7.34 ms ✅ -0.14 ms (-1.90%) 1.00%
handshake_session_id_ring_1.3_ecdsap256_aes 6.88 ms 6.75 ms ✅ -0.13 ms (-1.85%) 1.09%
handshake_session_id_ring_1.3_ecdsap256_chacha 6.84 ms 6.72 ms ✅ -0.13 ms (-1.85%) 1.15%
transfer_no_resume_ring_1.3_ecdsap384_chacha 16.34 ms 16.04 ms ✅ -0.29 ms (-1.78%) 1.36%
handshake_tickets_ring_1.3_ecdsap256_aes 6.90 ms 6.78 ms ✅ -0.12 ms (-1.71%) 1.01%
handshake_tickets_ring_1.3_ecdsap256_chacha 6.86 ms 6.74 ms ✅ -0.12 ms (-1.70%) 1.17%
handshake_session_id_ring_1.3_ecdsap384_aes 9.98 ms 9.84 ms ✅ -0.13 ms (-1.35%) 1.00%
handshake_session_id_ring_1.3_ecdsap384_chacha 9.94 ms 9.82 ms ✅ -0.13 ms (-1.27%) 1.00%
handshake_tickets_ring_1.3_ecdsap384_aes 10.00 ms 9.88 ms ✅ -0.12 ms (-1.25%) 1.00%
handshake_tickets_ring_1.3_ecdsap384_chacha 9.96 ms 9.84 ms ✅ -0.12 ms (-1.21%) 1.01%

Other differences

Click to expand
Scenario Baseline Candidate Diff Threshold
handshake_no_resume_ring_1.3_rsa_aes 1.08 ms 1.09 ms 0.01 ms (0.85%) 1.00%
handshake_no_resume_ring_1.3_rsa_chacha 1.08 ms 1.09 ms 0.01 ms (0.73%) 1.00%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_aes 473.87 µs 471.77 µs -2.10 µs (-0.44%) 2.58%
handshake_no_resume_ring_1.3_ecdsap256_chacha 503.73 µs 501.51 µs -2.22 µs (-0.44%) 2.34%
handshake_no_resume_ring_1.2_rsa_aes 1.07 ms 1.07 ms -0.00 ms (-0.31%) 1.00%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_chacha 472.45 µs 471.25 µs -1.20 µs (-0.25%) 2.86%
handshake_no_resume_ring_1.3_ecdsap256_aes 505.58 µs 504.39 µs -1.19 µs (-0.24%) 1.84%
handshake_no_resume_aws_lc_rs_1.2_rsa_aes 1.36 ms 1.36 ms -0.00 ms (-0.20%) 1.00%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_aes 1.19 ms 1.18 ms -0.00 ms (-0.16%) 1.04%
handshake_no_resume_ring_1.3_ecdsap384_chacha 3.61 ms 3.61 ms -0.00 ms (-0.13%) 1.00%
handshake_no_resume_ring_1.3_ecdsap384_aes 3.61 ms 3.61 ms -0.00 ms (-0.08%) 1.00%
handshake_no_resume_aws_lc_rs_1.3_rsa_chacha 1.40 ms 1.40 ms -0.00 ms (-0.05%) 1.00%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_chacha 1.18 ms 1.18 ms -0.00 ms (-0.05%) 1.16%
handshake_no_resume_aws_lc_rs_1.3_rsa_aes 1.41 ms 1.41 ms 0.00 ms (0.04%) 1.00%

Additional information

Historical results

Checkout details:

@codecov Codecov
Copy link

codecov bot commented Mar 14, 2024 •
edited
Loading

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.26%. Comparing base (9a91184) to head (ffe3040).
Report is 2 commits behind head on main.

Additional details and impacted files 
@@           Coverage Diff           @@
##             main    #1849   +/-   ##
=======================================
  Coverage   95.26%   95.26%           
=======================================
  Files          85       85           
  Lines       18960    18960           
=======================================
  Hits        18063    18063           
  Misses        897      897

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

djc
djc reviewed Mar 14, 2024
View reviewed changes
Cargo.toml Outdated Show resolved Hide resolved
@ctz
Enable lto for bench profile
ffe3040 
Gives:

- 11-12% improvement on bulk receiving benchmarks.
- 2-5% improvement on handshake benchmarks

Use this when building the `bench` tool.
@ctz ctz enabled auto-merge March 14, 2024 15:51
@ctz ctz added this pull request to the merge queue Mar 14, 2024
Merged via the queue into main with commit 134e43f Mar 14, 2024
46 checks passed
@ctz ctz deleted the jbp-bench-jemalloc branch March 14, 2024 16:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@djc djc djc approved these changes

@cpu cpu cpu approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Support jemalloc for benchmarks
3 participants
@ctz @djc @cpu