perf: Store small bitsets inline in DenseBitSet

[xmakro]

DenseBitSet heap-allocated its words even for a tiny domain, and dataflow allocates one per basic block. Storing the words in a SmallVec<[Word; 2]> keeps bitsets up to 128 bits inline, so small ones no longer allocate.

[Kobzol]

@bors try @rust-timer queue

[rust-bors]

☀️ Try build successful (CI)
Build commit: 98ed799 (98ed7998665e79900491b517795dd93238acee1d)
Base parent: 8ab9fdf (8ab9fdff5a91b9f2b5ed57fb0275452d9a0d0280)

[panstromek]

Note that this is how it was originally stored, but it's been removed in #147644 for performance reasons.

There's been a few more experiments for DenseBitSet going on recently, notably: #153589, #141325 and #157024

These all try to address the similar problem. In my opinion, the Raw bit set from Zalathar is closest to what we'd probably want to do. I'm skeptical that the solutions that try to solve it internally will work, because they introduce branch to every access that's troublesome to optimize out.

What I think is more fruitful experiment (and I want to try at some point, if nobody else beat me to it) is to make the BitSet generic over the storage and move the allocation decision to the creation time, such that we can only pass &mut [Word] into it as a storage. This will be more significant change, but probably doable incrementally if we leave Vec<Word> as a default storage parameter.

[rust-timer]

Finished benchmarking commit (98ed799): comparison URL.

Overall result: ❌✅ regressions and improvements - please read:

Benchmarking means the PR may be perf-sensitive. It's automatically marked not fit for rolling up. Overriding is possible but disadvised: it risks changing compiler perf.

Next, please: If you can, justify the regressions found in this try perf run in writing along with @rustbot label: +perf-regression-triaged. If not, fix the regressions and do another perf run. Neutral or positive results will clear the label automatically.

@bors rollup=never rustc-perf
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

	mean	range	count
Regressions ❌ (primary)	0.5%	[0.2%, 0.8%]	31
Regressions ❌ (secondary)	0.5%	[0.1%, 1.3%]	43
Improvements ✅ (primary)	-0.3%	[-0.6%, -0.2%]	5
Improvements ✅ (secondary)	-0.5%	[-0.7%, -0.1%]	12
All ❌✅ (primary)	0.3%	[-0.6%, 0.8%]	36

Max RSS (memory usage)

Results (primary 0.7%, secondary 1.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	0.7%	[0.7%, 0.7%]	1
Regressions ❌ (secondary)	3.2%	[2.0%, 6.7%]	5
Improvements ✅ (primary)	-	-	0
Improvements ✅ (secondary)	-0.6%	[-0.9%, -0.4%]	7
All ❌✅ (primary)	0.7%	[0.7%, 0.7%]	1

Cycles

Results (primary -0.2%, secondary -0.3%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

	mean	range	count
Regressions ❌ (primary)	1.2%	[0.9%, 1.5%]	3
Regressions ❌ (secondary)	0.6%	[0.5%, 0.7%]	7
Improvements ✅ (primary)	-1.1%	[-1.9%, -0.6%]	5
Improvements ✅ (secondary)	-1.3%	[-2.8%, -0.5%]	6
All ❌✅ (primary)	-0.2%	[-1.9%, 1.5%]	8

Binary size

This perf run didn't have relevant results for this metric.

Bootstrap: 492.09s -> 488.725s (-0.68%)
Artifact size: 390.63 MiB -> 390.70 MiB (0.02%)

[xmakro]

Thanks @panstromek, I had profiled without jemalloc, which lead to the wrong result

[panstromek]

No problem, sometimes it's also worth trying again, so don't worry about it. The change to Vec from SmallVec was itself a reversal of previous Vec -> SmallVec optimization. Both led to an improvement at the time of their landing. Things change around how these types are used, so the optimal decision could be different this time again.