[benchmark] Rework SuperChars

[milseman]

Rework SuperChars into something that:

• Doesn't crash in asserts configurations (i.e. has valid Characters)
• Actually runs (i.e. use blackHole on comparison result)
• Runs in around 1ms (it's cubic complexity in payload size)

[milseman]

@swift-ci please test

[milseman]

@swift-ci please benchmark

[milseman]

@palimondo, since I'm updating this benchmark, any recommendations or other things that should be changed?

[swift-ci]

Build comment file:

Performance: -O

TEST	OLD	NEW	DELTA	RATIO
Improvement
DataAppendDataLargeToLarge	51400	37800	-26.5%	1.36x (?)
Added
SuperChars2	1770	1821	1787	—
Removed
SuperChars	3	3	3	—

Code size: -O

TEST	OLD	NEW	DELTA	RATIO
Regression
SuperChars.o	1124	1478	+31.5%	0.76x

Performance: -Osize

TEST	MIN	MAX	MEAN	MAX_RSS
Added
SuperChars2	1778	1865	1807	—
Removed
SuperChars	3	3	3	—

Code size: -Osize

TEST	OLD	NEW	DELTA	RATIO
Regression
SuperChars.o	1246	1600	+28.4%	0.78x

Performance: -Onone

TEST	MIN	MAX	MEAN	MAX_RSS
Added
SuperChars2	11915	12016	11961	—
Removed
SuperChars	126814	126978	126917	—

✅	Benchmark Check Report
⚠️⏱	SuperChars2 execution took at least 1766 μs. Decrease the workload of SuperChars2 by a factor of 2 (10), to be less than 1000 μs.

How to read the data

The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview

  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 12-Core Intel Xeon E5
  Processor Speed: 2.7 GHz
  Number of Processors: 1
  Total Number of Cores: 12
  L2 Cache (per Core): 256 KB
  L3 Cache: 30 MB
  Memory: 64 GB

--------------

[palimondo]

LGTM, but I took the liberty to lower the workload a bit further.

My local results on 2008 MBP, before (779252a):

TEST	MIN	Q1	MED	Q3	MAX	MAX_RSS(B)
SuperChars2	4515	4574	4610	4757	5058	49152

after (a30b774):

TEST	MIN	Q1	MED	Q3	MAX	MAX_RSS(B)
SuperChars2	1320	1326	1331	1339	1362	49152

[palimondo]

@swift-ci please benchmark

[palimondo]

@swift-ci please smoke test

[swift-ci]

Build comment file:

Performance: -O

TEST	MIN	MAX	MEAN	MAX_RSS
Added
SuperChars2	526	528	527	—
Removed
SuperChars	3	3	3	—

Code size: -O

TEST	OLD	NEW	DELTA	RATIO
Regression
SuperChars.o	1124	1478	+31.5%	0.76x

Performance: -Osize

TEST	MIN	MAX	MEAN	MAX_RSS
Added
SuperChars2	527	529	528	—
Removed
SuperChars	3	3	3	—

Code size: -Osize

TEST	OLD	NEW	DELTA	RATIO
Regression
SuperChars.o	1246	1600	+28.4%	0.78x

Performance: -Onone

TEST	OLD	NEW	DELTA	RATIO
Regression
StringBuilderWithLongSubstring	3190	3520	+10.3%	0.91x (?)
Added
SuperChars2	2376	2430	2394	—
Removed
SuperChars	115876	116232	116043	—

✅	Benchmark Check Report

How to read the data

The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview

  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 12-Core Intel Xeon E5
  Processor Speed: 2.7 GHz
  Number of Processors: 1
  Total Number of Cores: 12
  L2 Cache (per Core): 256 KB
  L3 Cache: 30 MB
  Memory: 64 GB

--------------