benchmarks: disable some unstable benchmarks

[eeckstein]

and scale a substring test

[eeckstein]

@swift-ci benchmark

[eeckstein]

@swift-ci smoke test

[swift-ci]

Performance (x86_64): -O

Regression	OLD	NEW	DELTA	RATIO
FlattenListFlatMap	5294	5997	+13.3%	0.88x (?)
Improvement	OLD	NEW	DELTA	RATIO
DataCreateEmpty	170	150	-11.8%	1.13x (?)
Added	MIN	MAX	MEAN	MAX_RSS
SubstringFromLongString2	99	104	101	—
Removed	MIN	MAX	MEAN	MAX_RSS
ArrayAppend	690	690	690	—
DataAppendDataLargeToLarge	81800	82400	82133	—
Memset	202	206	203	—
SubstringFromLongString	9	10	10	—

Code size: -O

Regression	OLD	NEW	DELTA	RATIO
ArrayAppend.o	24777	25081	+1.2%	0.99x

Performance (x86_64): -Osize

Regression	OLD	NEW	DELTA	RATIO
DataCountSmall	25	28	+12.0%	0.89x (?)
SubstringComparable	10	11	+10.0%	0.91x (?)
DataAccessBytesMedium	75	82	+9.3%	0.91x (?)
Improvement	OLD	NEW	DELTA	RATIO
FlattenListFlatMap	5819	5242	-9.9%	1.11x (?)
NSStringConversion.MutableCopy.Rebridge.UTF8	761	711	-6.6%	1.07x (?)
Added	MIN	MAX	MEAN	MAX_RSS
SubstringFromLongString2	100	101	100	—
Removed	MIN	MAX	MEAN	MAX_RSS
ArrayAppend	760	760	760	—
DataAppendDataLargeToLarge	53200	53800	53467	—
Memset	290	292	291	—
SubstringFromLongString	10	10	10	—

Code size: -Osize

Regression	OLD	NEW	DELTA	RATIO
ArrayAppend.o	22249	22501	+1.1%	0.99x

Performance (x86_64): -Onone

Improvement	OLD	NEW	DELTA	RATIO
StringToDataMedium	6000	5450	-9.2%	1.10x (?)
NSStringConversion.MutableCopy.Rebridge.Medium	718	671	-6.5%	1.07x (?)
Added	MIN	MAX	MEAN	MAX_RSS
SubstringFromLongString2	671	672	671	—
Removed	MIN	MAX	MEAN	MAX_RSS
ArrayAppend	6560	6570	6567	—
DataAppendDataLargeToLarge	86200	369800	181200	—
Memset	16383	16661	16476	—
SubstringFromLongString	68	72	69	—

Code size: -swiftlibs

✅	Benchmark Check Report

How to read the data

The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview

  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 8-Core Intel Xeon E5
  Processor Speed: 3 GHz
  Number of Processors: 1
  Total Number of Cores: 8
  L2 Cache (per Core): 256 KB
  L3 Cache: 25 MB
  Memory: 64 GB