2024-06-23.md

2024-06-23 Triage Log

Mostly a number of improvements driven by MIR inliner improvements, with a small number benchmarks having a significant regression due to improvements in sort algorithms, which are runtime improvements at the cost of usually slight or neutral compile time regressions, with outliers in a few cases.

Triage done by @simulacrum. Revision range: c2932aaf..c3d7fb39

Summary:

(instructions:u)	mean	range	count
Regressions ❌ (primary)	2.6%	[0.2%, 16.4%]	36
Regressions ❌ (secondary)	1.4%	[0.1%, 5.0%]	31
Improvements ✅ (primary)	-0.8%	[-1.6%, -0.2%]	184
Improvements ✅ (secondary)	-0.9%	[-3.4%, -0.3%]	138
All ❌✅ (primary)	-0.2%	[-1.6%, 16.4%]	220

4 Regressions, 5 Improvements, 3 Mixed; 2 of them in rollups 48 artifact comparisons made in total

Regressions

Rollup of 10 pull requests #126655 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.3%	[0.2%, 0.3%]	6
Regressions ❌ (secondary)	0.5%	[0.2%, 0.9%]	8
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.3%	[0.2%, 0.3%]	6

Slight instruction count regressions, but mostly in somewhat noisy benchmarks. It looks like performance went back to previous levels in the next few commits.

Replace sort implementations #124032 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	2.1%	[0.2%, 18.0%]	51
Regressions ❌ (secondary)	1.6%	[0.4%, 3.9%]	8
Improvements ✅ (primary)	-0.3%	[-0.3%, -0.3%]	1
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	2.0%	[-0.3%, 18.0%]	52

This change was extensively benchmarked, and the regressions were known ahead of time. Runtime improvements are considered worth the regressions, which are limited to just a few benchmarks. (Extensive effort was also put in to minimize the compile time and binary size costs).

Properly gate safe keyword in pre-expansion #126757 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.2%	[0.2%, 0.3%]	4
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.2%	[0.2%, 0.3%]	4

Looks like a minimal regression (maybe spurious/bimodality) and the change is clearly needed. Marking as triaged.

Generalize {Rc,Arc}::make_mut() to unsized types. #116113 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.8%	[0.3%, 2.2%]	13
Regressions ❌ (secondary)	0.9%	[0.9%, 0.9%]	1
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	0.8%	[0.3%, 2.2%]	13

Regressions are mostly in doc benchmarks, seem likely to be just new docs due to extra stuff in the standard library.

Improvements

Migrate extern-flag-fun, incremental-debugger-visualiser and incremental-session-fail run-make tests to rmake.rs #126490 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.3%	[-0.4%, -0.3%]	6
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-0.3%	[-0.4%, -0.3%]	6

Rework feature(precise_capturing) to represent use<...> as a syntactical bound #126049 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.2%	[-0.2%, -0.2%]	7
Improvements ✅ (secondary)	-0.4%	[-0.6%, -0.2%]	12
All ❌✅ (primary)	-0.2%	[-0.2%, -0.2%]	7

Fix duplicated attributes on nonterminal expressions #126678 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.4%	[-0.5%, -0.2%]	7
Improvements ✅ (secondary)	-0.7%	[-0.7%, -0.6%]	3
All ❌✅ (primary)	-0.4%	[-0.5%, -0.2%]	7

Trying to address an incremental compilation issues #126409 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.3%	[-0.4%, -0.2%]	21
Improvements ✅ (secondary)	-0.3%	[-0.3%, -0.2%]	15
All ❌✅ (primary)	-0.3%	[-0.4%, -0.2%]	21

Rollup of 3 pull requests #126838 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	-	-	0
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-0.4%	[-0.9%, -0.2%]	50
Improvements ✅ (secondary)	-0.4%	[-0.9%, -0.2%]	19
All ❌✅ (primary)	-0.4%	[-0.9%, -0.2%]	50

Mixed

[perf] More span update benchmarking #126591 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.3%	[0.3%, 0.4%]	5
Regressions ❌ (secondary)	2.3%	[1.4%, 3.6%]	6
Improvements ✅ (primary)	-0.3%	[-0.3%, -0.3%]	1
Improvements ✅ (secondary)	-0.3%	[-0.4%, -0.2%]	14
All ❌✅ (primary)	0.2%	[-0.3%, 0.4%]	6

Triaged to be mostly just inlining noise: rust-lang/rust#126591 (comment)

Account for things that optimize out in inlining costs #126578 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	1.4%	[0.4%, 2.0%]	6
Regressions ❌ (secondary)	1.4%	[0.3%, 4.1%]	16
Improvements ✅ (primary)	-0.6%	[-1.5%, -0.2%]	183
Improvements ✅ (secondary)	-0.9%	[-4.6%, -0.2%]	121
All ❌✅ (primary)	-0.5%	[-1.5%, 2.0%]	189

Mostly an improvement for the majority of benchmarks, including some slight runtime improvements.

More ptr metadata gvn #126541 (Comparison Link)

(instructions:u)	mean	range	count
Regressions ❌ (primary)	0.2%	[0.2%, 0.2%]	1
Regressions ❌ (secondary)	-	-	0
Improvements ✅ (primary)	-1.0%	[-1.4%, -0.6%]	2
Improvements ✅ (secondary)	-	-	0
All ❌✅ (primary)	-0.6%	[-1.4%, 0.2%]	3

Slight improvements on full builds, slight regression in incremental for cargo. In general fairly neutral, only a few benchmarks affected.