github actions: standardise on ubuntu-latest

[ctz]

Prior to this we had a mix of ubuntu-20.04 and ubuntu-latest. I'd like to have just one, so I can use act locally to do test builds without having multiple docker images.

This will probably trip up the branch protection rules.

[rustls-benchmarking]

Benchmark results

Instruction counts

Significant differences

There are no significant instruction count differences

Other differences

Click to expand

Scenario	Baseline	Candidate	Diff	Threshold
handshake_no_resume_aws_lc_rs_1.3_rsa_aes_server	12703268	12651989	-51279 (-0.40%)	0.80%
handshake_session_id_aws_lc_rs_1.3_rsa_aes_server	32613428	32705156	91728 (0.28%)	0.49%
transfer_no_resume_aws_lc_rs_1.3_rsa_aes_server	57129901	57270668	140767 (0.25%)	0.43%
handshake_tickets_aws_lc_rs_1.2_rsa_aes_server	4606032	4594894	-11138 (-0.24%)	3.21%
handshake_tickets_aws_lc_rs_1.3_rsa_chacha_server	32894383	32830386	-63997 (-0.19%)	0.43%
handshake_no_resume_aws_lc_rs_1.3_rsa_chacha_server	12695101	12719677	24576 (0.19%)	0.63%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_aes_client	8675850	8659637	-16213 (-0.19%)	0.78%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_client	8658894	8674622	15728 (0.18%)	1.64%
handshake_tickets_aws_lc_rs_1.2_rsa_aes_client	4562091	4570377	8286 (0.18%)	1.03%
handshake_no_resume_aws_lc_rs_1.2_rsa_aes_server	12253071	12273935	20864 (0.17%)	0.81%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_aes_client	30499066	30544104	45038 (0.15%)	0.49%
handshake_no_resume_ring_1.3_ecdsap256_chacha_server	2130391	2128104	-2287 (-0.11%)	0.20%
handshake_tickets_aws_lc_rs_1.3_rsa_aes_server	32879464	32913821	34357 (0.10%)	0.79%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_aes_client	3340292	3343153	2861 (0.09%)	0.23%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_aes_server	32654304	32629036	-25268 (-0.08%)	0.20%
handshake_tickets_ring_1.2_rsa_aes_client	4747867	4744264	-3603 (-0.08%)	0.82%
handshake_session_id_ring_1.2_rsa_aes_client	4460544	4463793	3249 (0.07%)	0.50%
transfer_no_resume_aws_lc_rs_1.2_rsa_aes_server	57157561	57198900	41339 (0.07%)	0.38%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_chacha_server	32629507	32606472	-23035 (-0.07%)	0.43%
transfer_no_resume_ring_1.3_ecdsap256_aes_server	57131429	57094714	-36715 (-0.06%)	0.26%
handshake_tickets_ring_1.3_ecdsap384_chacha_server	43844096	43869624	25528 (0.06%)	0.29%
handshake_tickets_aws_lc_rs_1.3_rsa_aes_client	30731586	30714159	-17427 (-0.06%)	0.45%
handshake_tickets_ring_1.3_rsa_aes_server	43937187	43961610	24423 (0.06%)	0.20%
handshake_session_id_aws_lc_rs_1.2_rsa_aes_client	4224321	4222085	-2236 (-0.05%)	1.25%
handshake_tickets_ring_1.3_ecdsap384_aes_server	43904003	43882921	-21082 (-0.05%)	0.22%
handshake_no_resume_ring_1.3_ecdsap256_aes_client	3893973	3892222	-1751 (-0.04%)	0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_chacha_client	30510031	30496495	-13536 (-0.04%)	0.72%
handshake_session_id_aws_lc_rs_1.3_rsa_aes_client	30512438	30525839	13401 (0.04%)	0.42%
handshake_session_id_ring_1.3_ecdsap384_aes_server	43648887	43666512	17625 (0.04%)	0.20%
handshake_tickets_ring_1.3_rsa_chacha_client	42368757	42352680	-16077 (-0.04%)	0.20%
transfer_no_resume_ring_1.3_ecdsap256_chacha_server	91294460	91259961	-34499 (-0.04%)	0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_aes_server	57157911	57179306	21395 (0.04%)	0.20%
handshake_session_id_aws_lc_rs_1.3_rsa_chacha_server	32607652	32619775	12123 (0.04%)	0.44%
handshake_tickets_aws_lc_rs_1.3_rsa_chacha_client	30684035	30695375	11340 (0.04%)	0.41%
transfer_no_resume_ring_1.3_rsa_aes_server	57087791	57067632	-20159 (-0.04%)	0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_aes_server	1881862	1882494	632 (0.03%)	0.20%
transfer_no_resume_ring_1.3_ecdsap384_aes_server	57103372	57084501	-18871 (-0.03%)	0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_chacha_client	30505052	30495139	-9913 (-0.03%)	0.50%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_aes_server	32874812	32864335	-10477 (-0.03%)	0.39%
handshake_session_id_aws_lc_rs_1.3_rsa_chacha_client	30499266	30508820	9554 (0.03%)	0.45%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_aes_client	30318732	30327236	8504 (0.03%)	0.47%
handshake_no_resume_ring_1.3_ecdsap384_chacha_server	13739862	13736196	-3666 (-0.03%)	0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_aes_client	30335304	30343186	7882 (0.03%)	0.48%
handshake_session_id_ring_1.3_rsa_aes_client	42204688	42215632	10944 (0.03%)	0.20%
handshake_tickets_ring_1.2_rsa_aes_server	4845792	4846922	1130 (0.02%)	0.67%
handshake_tickets_ring_1.3_ecdsap256_aes_server	43913276	43903049	-10227 (-0.02%)	0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_aes_client	57959293	57972712	13419 (0.02%)	0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_chacha_server	91332799	91311853	-20946 (-0.02%)	0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_server	1885598	1886026	428 (0.02%)	0.20%
handshake_session_id_ring_1.3_ecdsap384_chacha_server	43588336	43596675	8339 (0.02%)	0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_aes_server	32879523	32873388	-6135 (-0.02%)	0.31%
handshake_session_id_ring_1.3_rsa_aes_server	43663960	43656254	-7706 (-0.02%)	0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_client	3344909	3345499	590 (0.02%)	0.28%
handshake_session_id_aws_lc_rs_1.2_rsa_aes_server	4110091	4110782	691 (0.02%)	3.86%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_chacha_server	32849456	32854846	5390 (0.02%)	0.39%
handshake_tickets_ring_1.3_ecdsap384_aes_client	42178062	42184947	6885 (0.02%)	0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_chacha_server	32612997	32618267	5270 (0.02%)	0.39%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_aes_server	4260638	4261307	669 (0.02%)	0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_chacha_client	30310536	30314854	4318 (0.01%)	0.41%
handshake_tickets_ring_1.3_rsa_aes_client	42383403	42389197	5794 (0.01%)	0.20%
handshake_no_resume_ring_1.3_ecdsap384_chacha_client	35454690	35450654	-4036 (-0.01%)	0.20%
handshake_session_id_ring_1.3_ecdsap256_aes_server	43661798	43656884	-4914 (-0.01%)	0.20%
handshake_session_id_ring_1.3_rsa_chacha_client	42159675	42155435	-4240 (-0.01%)	0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_client	92450224	92440957	-9267 (-0.01%)	0.20%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_server	4264871	4265293	422 (0.01%)	0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_chacha_client	30311523	30314520	2997 (0.01%)	0.51%
handshake_session_id_ring_1.3_rsa_chacha_server	43597626	43593630	-3996 (-0.01%)	0.20%
handshake_session_id_ring_1.3_ecdsap256_chacha_server	43584246	43587609	3363 (0.01%)	0.20%
handshake_no_resume_ring_1.3_rsa_chacha_client	4547555	4547891	336 (0.01%)	0.20%
handshake_tickets_ring_1.3_ecdsap384_chacha_client	42144420	42147445	3025 (0.01%)	0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_aes_client	30510631	30508570	-2061 (-0.01%)	0.40%
handshake_session_id_ring_1.3_ecdsap256_chacha_client	41970491	41967847	-2644 (-0.01%)	0.20%
handshake_session_id_ring_1.3_ecdsap256_aes_client	42015256	42012908	-2348 (-0.01%)	0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_chacha_client	92447256	92452209	4953 (0.01%)	0.20%
handshake_tickets_ring_1.3_ecdsap256_aes_client	42185195	42187386	2191 (0.01%)	0.20%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_chacha_server	32836666	32835028	-1638 (-0.00%)	0.47%
handshake_session_id_ring_1.2_rsa_aes_server	4378755	4378538	-217 (-0.00%)	0.60%
handshake_tickets_ring_1.3_ecdsap256_chacha_server	43842846	43840760	-2086 (-0.00%)	0.30%
transfer_no_resume_ring_1.3_ecdsap384_chacha_client	92388343	92392596	4253 (0.00%)	0.20%
transfer_no_resume_ring_1.3_ecdsap384_chacha_server	91255949	91260053	4104 (0.00%)	0.20%
handshake_no_resume_aws_lc_rs_1.2_rsa_aes_client	3153241	3153379	138 (0.00%)	0.20%
handshake_session_id_ring_1.3_ecdsap384_chacha_client	41971080	41969290	-1790 (-0.00%)	0.20%
handshake_no_resume_ring_1.3_rsa_chacha_server	12249631	12250095	464 (0.00%)	0.20%
handshake_no_resume_ring_1.3_ecdsap256_aes_server	2126088	2126009	-79 (-0.00%)	0.20%
handshake_no_resume_ring_1.2_rsa_aes_client	4441573	4441737	164 (0.00%)	0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_aes_client	57949998	57947909	-2089 (-0.00%)	0.20%
handshake_tickets_ring_1.3_rsa_chacha_server	43907411	43905829	-1582 (-0.00%)	0.20%
transfer_no_resume_ring_1.3_ecdsap256_aes_client	57956108	57958042	1934 (0.00%)	0.20%
handshake_no_resume_aws_lc_rs_1.3_rsa_aes_client	3356248	3356139	-109 (-0.00%)	0.20%
handshake_no_resume_aws_lc_rs_1.3_rsa_chacha_client	3367428	3367323	-105 (-0.00%)	0.20%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_aes_server	32638117	32639059	942 (0.00%)	0.32%
handshake_no_resume_ring_1.3_rsa_aes_server	12239473	12239814	341 (0.00%)	0.20%
handshake_no_resume_ring_1.3_ecdsap384_aes_server	13736361	13736009	-352 (-0.00%)	0.20%
transfer_no_resume_ring_1.3_ecdsap384_aes_client	57947823	57949234	1411 (0.00%)	0.20%
handshake_no_resume_ring_1.2_rsa_aes_server	12045632	12045893	261 (0.00%)	0.20%
handshake_no_resume_ring_1.3_ecdsap384_aes_client	35451953	35451538	-415 (-0.00%)	0.20%
handshake_no_resume_ring_1.3_rsa_aes_client	4537755	4537808	53 (0.00%)	0.20%
handshake_tickets_ring_1.3_ecdsap256_chacha_client	42146222	42145779	-443 (-0.00%)	0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_client	92417644	92416697	-947 (-0.00%)	0.20%
transfer_no_resume_ring_1.3_rsa_aes_client	57951516	57951995	479 (0.00%)	0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_aes_server	57178177	57177715	-462 (-0.00%)	0.20%
transfer_no_resume_ring_1.3_rsa_chacha_client	92391736	92391177	-559 (-0.00%)	0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_chacha_server	91327794	91327393	-401 (-0.00%)	0.20%
transfer_no_resume_ring_1.3_ecdsap256_chacha_client	92389767	92390165	398 (0.00%)	0.20%
transfer_no_resume_aws_lc_rs_1.3_rsa_aes_client	57992722	57992950	228 (0.00%)	0.20%
handshake_session_id_ring_1.3_ecdsap384_aes_client	42016994	42017126	132 (0.00%)	0.20%
transfer_no_resume_ring_1.3_rsa_chacha_server	91252294	91252087	-207 (-0.00%)	0.20%
transfer_no_resume_ring_1.2_rsa_aes_client	57812391	57812486	95 (0.00%)	0.20%
transfer_no_resume_ring_1.2_rsa_aes_server	56972170	56972211	41 (0.00%)	0.20%
handshake_no_resume_ring_1.3_ecdsap256_chacha_client	3896847	3896845	-2 (-0.00%)	0.20%
transfer_no_resume_aws_lc_rs_1.2_rsa_aes_client	68390287	68390269	-18 (-0.00%)	0.20%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_chacha_server	91348079	91348080	1 (0.00%)	0.20%

Wall-time

Significant differences

There are no significant wall-time differences

Other differences

Click to expand

Scenario	Baseline	Candidate	Diff	Threshold
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_aes	5.68 ms	5.62 ms	-0.06 ms (-1.10%)	5.28%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_aes	4.96 ms	4.91 ms	-0.05 ms (-1.04%)	6.75%
transfer_no_resume_aws_lc_rs_1.3_rsa_aes	5.90 ms	5.85 ms	-0.05 ms (-0.88%)	4.55%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_chacha	6.12 ms	6.07 ms	-0.05 ms (-0.79%)	2.27%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_chacha	5.42 ms	5.37 ms	-0.04 ms (-0.78%)	2.36%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_chacha	5.45 ms	5.41 ms	-0.04 ms (-0.75%)	2.32%
handshake_session_id_aws_lc_rs_1.3_rsa_chacha	6.42 ms	6.37 ms	-0.04 ms (-0.70%)	1.81%
handshake_tickets_aws_lc_rs_1.3_ecdsap256_aes	5.44 ms	5.40 ms	-0.04 ms (-0.67%)	2.60%
handshake_session_id_aws_lc_rs_1.3_ecdsap256_aes	5.42 ms	5.39 ms	-0.04 ms (-0.67%)	2.70%
transfer_no_resume_aws_lc_rs_1.2_rsa_aes	5.89 ms	5.85 ms	-0.04 ms (-0.65%)	3.25%
handshake_session_id_aws_lc_rs_1.3_ecdsap384_aes	6.13 ms	6.09 ms	-0.04 ms (-0.64%)	2.40%
transfer_no_resume_ring_1.3_ecdsap256_aes	6.75 ms	6.71 ms	-0.04 ms (-0.63%)	3.73%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_aes	6.16 ms	6.12 ms	-0.04 ms (-0.62%)	1.98%
handshake_tickets_aws_lc_rs_1.3_ecdsap384_chacha	6.16 ms	6.12 ms	-0.04 ms (-0.62%)	1.60%
handshake_no_resume_ring_1.3_rsa_aes	1.08 ms	1.09 ms	0.01 ms (0.61%)	1.18%
handshake_tickets_aws_lc_rs_1.3_rsa_chacha	6.45 ms	6.41 ms	-0.04 ms (-0.60%)	1.68%
transfer_no_resume_ring_1.3_rsa_aes	7.33 ms	7.29 ms	-0.04 ms (-0.59%)	3.20%
transfer_no_resume_ring_1.2_rsa_aes	7.24 ms	7.20 ms	-0.04 ms (-0.59%)	2.60%
handshake_session_id_aws_lc_rs_1.3_rsa_aes	6.43 ms	6.39 ms	-0.03 ms (-0.53%)	2.28%
handshake_tickets_aws_lc_rs_1.3_rsa_aes	6.44 ms	6.41 ms	-0.03 ms (-0.47%)	2.22%
transfer_no_resume_ring_1.3_ecdsap384_aes	9.85 ms	9.82 ms	-0.04 ms (-0.40%)	2.85%
transfer_no_resume_aws_lc_rs_1.3_ecdsap256_chacha	13.45 ms	13.40 ms	-0.05 ms (-0.39%)	2.36%
transfer_no_resume_aws_lc_rs_1.3_ecdsap384_chacha	14.16 ms	14.11 ms	-0.05 ms (-0.38%)	2.15%
handshake_session_id_aws_lc_rs_1.2_rsa_aes	2.18 ms	2.17 ms	-0.01 ms (-0.36%)	2.23%
handshake_no_resume_ring_1.3_rsa_chacha	1.09 ms	1.09 ms	0.00 ms (0.35%)	1.30%
transfer_no_resume_aws_lc_rs_1.3_rsa_chacha	14.39 ms	14.34 ms	-0.05 ms (-0.32%)	1.64%
transfer_no_resume_ring_1.3_ecdsap256_chacha	13.46 ms	13.42 ms	-0.04 ms (-0.32%)	2.24%
transfer_no_resume_ring_1.3_ecdsap384_chacha	16.57 ms	16.52 ms	-0.05 ms (-0.28%)	1.90%
handshake_no_resume_aws_lc_rs_1.3_rsa_chacha	1.41 ms	1.41 ms	-0.00 ms (-0.21%)	1.00%
transfer_no_resume_ring_1.3_rsa_chacha	14.03 ms	14.00 ms	-0.03 ms (-0.20%)	1.87%
handshake_tickets_ring_1.3_ecdsap256_aes	6.98 ms	6.96 ms	-0.01 ms (-0.18%)	1.00%
handshake_tickets_aws_lc_rs_1.2_rsa_aes	2.35 ms	2.34 ms	-0.00 ms (-0.18%)	2.01%
handshake_no_resume_ring_1.3_ecdsap384_aes	3.61 ms	3.62 ms	0.01 ms (0.17%)	1.00%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_aes	1.19 ms	1.19 ms	-0.00 ms (-0.17%)	1.00%
handshake_no_resume_ring_1.3_ecdsap256_aes	509.41 µs	508.68 µs	-0.73 µs (-0.14%)	2.48%
handshake_session_id_ring_1.2_rsa_aes	1.76 ms	1.76 ms	-0.00 ms (-0.14%)	2.12%
handshake_tickets_ring_1.2_rsa_aes	1.84 ms	1.84 ms	-0.00 ms (-0.13%)	2.44%
handshake_tickets_ring_1.3_ecdsap256_chacha	6.92 ms	6.91 ms	-0.01 ms (-0.13%)	1.13%
handshake_no_resume_ring_1.3_ecdsap256_chacha	507.64 µs	507.08 µs	-0.56 µs (-0.11%)	2.35%
handshake_no_resume_ring_1.3_ecdsap384_chacha	3.61 ms	3.62 ms	0.00 ms (0.10%)	1.00%
handshake_session_id_ring_1.3_ecdsap256_aes	6.95 ms	6.95 ms	-0.01 ms (-0.10%)	1.00%
handshake_tickets_ring_1.3_rsa_aes	7.61 ms	7.60 ms	-0.01 ms (-0.09%)	1.13%
handshake_session_id_ring_1.3_ecdsap256_chacha	6.91 ms	6.91 ms	-0.01 ms (-0.09%)	1.22%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_aes	477.65 µs	477.27 µs	-0.38 µs (-0.08%)	3.95%
handshake_no_resume_aws_lc_rs_1.2_rsa_aes	1.36 ms	1.36 ms	-0.00 ms (-0.07%)	1.39%
handshake_session_id_ring_1.3_ecdsap384_aes	10.05 ms	10.05 ms	-0.01 ms (-0.07%)	1.00%
handshake_tickets_ring_1.3_ecdsap384_aes	10.08 ms	10.07 ms	-0.01 ms (-0.07%)	1.00%
handshake_no_resume_aws_lc_rs_1.3_ecdsap256_chacha	477.63 µs	477.31 µs	-0.32 µs (-0.07%)	2.96%
handshake_tickets_ring_1.3_rsa_chacha	7.56 ms	7.55 ms	-0.00 ms (-0.07%)	1.19%
handshake_no_resume_aws_lc_rs_1.3_ecdsap384_chacha	1.18 ms	1.18 ms	-0.00 ms (-0.05%)	1.18%
handshake_session_id_ring_1.3_rsa_aes	7.59 ms	7.59 ms	-0.00 ms (-0.04%)	1.00%
handshake_no_resume_aws_lc_rs_1.3_rsa_aes	1.41 ms	1.41 ms	-0.00 ms (-0.03%)	1.00%
handshake_session_id_ring_1.3_rsa_chacha	7.54 ms	7.54 ms	-0.00 ms (-0.01%)	1.06%
handshake_tickets_ring_1.3_ecdsap384_chacha	10.02 ms	10.02 ms	0.00 ms (0.01%)	1.00%
handshake_no_resume_ring_1.2_rsa_aes	1.07 ms	1.07 ms	0.00 ms (0.01%)	1.20%
handshake_session_id_ring_1.3_ecdsap384_chacha	10.01 ms	10.01 ms	-0.00 ms (-0.01%)	1.00%

Additional information

Historical results

Checkout details:

• Base repo: https://github.com/rustls/rustls.git
• Base branch: main (5cc7157)
• Candidate repo: https://github.com/rustls/rustls.git
• Candidate branch: jbp-ubuntu-latest (94b48b2)

[cpu]

I'd like to have just one, so I can use act locally to do test builds without having multiple docker images.

Ah neat, I hadn't heard of this tool before: https://github.com/nektos/act

[ctz]

I'd like to have just one, so I can use act locally to do test builds without having multiple docker images.

Ah neat, I hadn't heard of this tool before: https://github.com/nektos/act

Yeah this mostly works for me -- you need to opt into the "full" images (which are 40-50GB, because they contain every version of every language runtime available). Though our "Check cross compilation targets" job because it's trying to use docker nested inside a docker container. But the remainder work.

[ctz]

Have updated the branch protection rules for this.