Add a benchmark for bridging String(contentsOfFile:...)

[Catfish-Man]

No description provided.

[Catfish-Man]

bleh

[Catfish-Man]

@swift-ci please smoke benchmark

[milseman]

If the intent is for this to be all-ASCII, add ASCII to the name.

[milseman]

Does Foundation have a mktemp equivalent?

[Catfish-Man]

The call in the other else branch is the best way, there's a clunkier version I could use on old OSs if it's a problem

[Catfish-Man]

@swift-ci please smoke test and merge

[Catfish-Man]

@swift-ci please smoke test and merge

[swift-ci]

Performance: -O

TEST	OLD	NEW	DELTA	RATIO
Regression
ObjectiveCBridgeStubFromNSDate	6400	7510	+17.3%	0.85x (?)
ObjectiveCBridgeStubToNSStringRef	124	141	+13.7%	0.88x (?)
ObjectiveCBridgeStubFromNSString	794	860	+8.3%	0.92x (?)
Improvement
ObjectiveCBridgeStubToNSDate2	1420	1220	-14.1%	1.16x (?)
Added
ObjectiveCBridgeASCIIStringFromFile	171	174	172	—

Code size: -O

TEST	OLD	NEW	DELTA	RATIO
Regression
ObjectiveCBridging.o	50060	52065	+4.0%	0.96x

Performance: -Osize

TEST	OLD	NEW	DELTA	RATIO
Regression
ObjectiveCBridgeStubDateAccess	228	257	+12.7%	0.89x
CharacterLiteralsLarge	100	111	+11.0%	0.90x (?)
Improvement
DataAppendDataLargeToLarge	51000	37200	-27.1%	1.37x (?)
Added
ObjectiveCBridgeASCIIStringFromFile	172	176	173	—

Code size: -Osize

TEST	OLD	NEW	DELTA	RATIO
Regression
ObjectiveCBridging.o	47715	49605	+4.0%	0.96x

Performance: -Onone

TEST	MIN	MAX	MEAN	MAX_RSS
Added
ObjectiveCBridgeASCIIStringFromFile	172	178	174	—

✅	Benchmark Check Report
⚠️🔤	ObjectiveCBridgeASCIIStringFromFile name is composed of 5 words. Split ObjectiveCBridgeASCIIStringFromFile name into dot-separated groups and variants. See http://bit.ly/BenchmarkNaming
⚠️Ⓜ️	ObjectiveCBridgeASCIIStringFromFile has very wide range of memory used between independent, repeated measurements. ObjectiveCBridgeASCIIStringFromFile mem_pages [i1, i2]: min=[109, 118] 𝚫=9 R=[25, 15]

How to read the data

The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview

  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 12-Core Intel Xeon E5
  Processor Speed: 2.7 GHz
  Number of Processors: 1
  Total Number of Cores: 12
  L2 Cache (per Core): 256 KB
  L3 Cache: 30 MB
  Memory: 64 GB

[palimondo]

Can somebody please explain the intention behind this benchmark?

All this does is run .utf8.count on a string that is in memory, but gets there in an extremely convoluted way during lazy initialization in the setUpFunction by writing a string to temporary file and loading it back in.

The setUpFunction get's executed only once before gathering multiple samples, not before every sample!

If you meant to test the performance of String(contentsOf:, encoding:), than you need to store the url in a global variable and move the last line from setup_ASCIIStringFromFile to the loop in run_ASCIIStringFromFile.

CC @Catfish-Man, @milseman, @eeckstein

[palimondo]

All this does is run .utf8.count on a string that is in memory…

I take that back! Looking at #24289 and #24386, I cannot believe the bridging can lead to this unbelievably convoluted and underperforming implementation behind the scenes. That is all kinds of wrong in my opinion. Improvements in #24386 lead to zero runtime of the ObjectiveCBridgeASCIIStringFromFile, so I believe this should not have been a benchmark in the first place, just a bug. I think the question still stands:

Can somebody please explain the intention behind this benchmark?

[milseman]

@Catfish-Man can probably provide more specific rationale, but there's a big issue with String(contentsOfFile:) even for all-ASCII files. CF does not believe the ASCII string is nul-terminated, so it will report back as being neither contiguous UTF-8 nor contiguous UTF-16. Instead, it is as though it were some arbitrary subclass of NSString and accessing the contents has to go through message sends fetching UTF-16 code units. I don't know if the URL-based one has similar issues (I suspect it would if loading from a file). This is likely part of the machinery in CF related to loading files.