[Perf -14%] System.Memory.ReadOnlySpan.IndexOfString (2)

[DrewScoggins]

Run Information

Architecture	x64
OS	ubuntu 18.04
Changes	diff

Regressions in System.Memory.ReadOnlySpan

Benchmark	Baseline	Test	Test/Base	Modality	Baseline Outlier
IndexOfString	27.38 ns	31.11 ns	1.14	Bimodal	False
IndexOfString	26.64 ns	30.73 ns	1.15	Bimodal	True

Related Issue on x64 Windows

[Perf -8%] System.Memory.ReadOnlySpan (2)

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Memory.ReadOnlySpan*';

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "X", comparisonType: Ordinal)

[25.975 ; 26.724) | @@@@@@@@@
[26.724 ; 27.949) | @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[27.949 ; 28.931) | @@@@@@@@@
[28.931 ; 30.259) | 
[30.259 ; 31.472) | @@@@@@@@@@@
[31.472 ; 32.697) | @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[32.697 ; 33.975) | @@@

System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだxだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "x", comparisonType: Ordinal)

[25.726 ; 26.775) | @@@@@@@@@@@@
[26.775 ; 27.991) | @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[27.991 ; 29.061) | @@@@@@@@@@
[29.061 ; 30.476) | 
[30.476 ; 31.554) | @@@@@@@@@@@@
[31.554 ; 32.770) | @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[32.770 ; 33.881) | @@@

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[Dotnet-GitSync-Bot]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[adamsitnik]

Overall in 3.1 vs 5.0 comparison there are 13 improvements and 1 minor regression. Closing

BenchmarkDotNet=v0.12.1.1405-nightly, OS=ubuntu 18.04
Intel Xeon CPU E5-1650 v4 3.60GHz, 1 CPU, 12 logical and 6 physical cores
.NET Core SDK=5.0.100-rc.1.20413.9
  [Host]     : .NET Core 3.1.6 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.31603), X64 RyuJIT
  Job-CJBAQZ : .NET Core 3.1.6 (CoreCLR 4.700.20.26901, CoreFX 4.700.20.31603), X64 RyuJIT

PowerPlanMode=00000000-0000-0000-0000-000000000000  Runtime=.NET Core 3.1  Arguments=/p:DebugType=portable  
Toolchain=netcoreapp3.1  IterationTime=250.0000 ms  MaxIterationCount=20  
MinIterationCount=15  WarmupCount=1  

Method	input	value	comparisonType	Mean	Error	StdDev	Median	Min	Max	Code Size	Gen 0	Gen 1	Gen 2	Allocated
IndexOfString	AAAAA5AAAA	5	InvariantCulture	49.56 ns	0.736 ns	0.689 ns	49.63 ns	48.55 ns	51.15 ns	1389 B	-	-	-	-
IndexOfString	AAAAAAAAAAAA(...)AAAAAAAAAAAA [1000]	X	Ordinal	38.95 ns	0.537 ns	0.502 ns	38.90 ns	38.19 ns	39.68 ns	1429 B	-	-	-	-
IndexOfString	AAAAAAAAAAAA(...)AAAAAAAAAAAA [100]	x	InvariantCultureIgnoreCase	181.68 ns	4.265 ns	4.912 ns	180.53 ns	175.41 ns	192.27 ns	1389 B	-	-	-	-
IndexOfString	AAAAAAAAAAAA(...)AAAAAAAAAAAA [100]	x	OrdinalIgnoreCase	577.15 ns	10.778 ns	10.585 ns	578.37 ns	559.83 ns	597.86 ns	1389 B	-	-	-	-
IndexOfString	ABCDE	c	InvariantCultureIgnoreCase	46.04 ns	0.917 ns	0.981 ns	45.95 ns	44.98 ns	47.97 ns	1389 B	-	-	-	-
IndexOfString	Hello Worldb(...)allylong!xyz [186]	w	OrdinalIgnoreCase	81.18 ns	1.591 ns	1.634 ns	80.67 ns	78.79 ns	85.56 ns	1389 B	-	-	-	-
IndexOfString	Hello Worldb(...)allylong!xyz [187]	~	Ordinal	21.66 ns	0.279 ns	0.261 ns	21.68 ns	21.24 ns	22.15 ns	1429 B	-	-	-	-
IndexOfString	Hello Worldbb(...)bbbbbbbbbbba! [47]	y	Ordinal	17.99 ns	0.128 ns	0.113 ns	18.03 ns	17.77 ns	18.14 ns	1429 B	-	-	-	-
IndexOfString	More Test's	Tests	OrdinalIgnoreCase	90.17 ns	1.553 ns	1.377 ns	90.03 ns	88.30 ns	93.08 ns	1389 B	-	-	-	-
IndexOfString	StrIng	string	OrdinalIgnoreCase	48.20 ns	0.202 ns	0.169 ns	48.15 ns	48.04 ns	48.62 ns	1389 B	-	-	-	-
IndexOfString	foobardzsdzs	rddzs	InvariantCulture	56.48 ns	0.187 ns	0.156 ns	56.46 ns	56.21 ns	56.77 ns	1389 B	-	-	-	-
IndexOfString	string1	string2	InvariantCulture	53.18 ns	0.054 ns	0.051 ns	53.18 ns	53.11 ns	53.29 ns	1389 B	-	-	-	-
IndexOfString	だ	た	InvariantCulture	4,238.15 ns	54.873 ns	51.329 ns	4,242.83 ns	4,144.78 ns	4,347.34 ns	1389 B	-	-	-	-
IndexOfString	だだだだだだだだだだだだ(...)だだだだだだだだだだだだ [100]	す	Ordinal	18.45 ns	0.055 ns	0.043 ns	18.44 ns	18.39 ns	18.52 ns	1429 B	-	-	-	-
IndexOfString	だだだだだだだだだだだだ(...)だだだだだだだだだだだだ [1000]	x	Ordinal	38.01 ns	0.060 ns	0.056 ns	38.02 ns	37.86 ns	38.09 ns	1429 B	-	-	-	-

BenchmarkDotNet=v0.12.1.1405-nightly, OS=ubuntu 18.04
Intel Xeon CPU E5-1650 v4 3.60GHz, 1 CPU, 12 logical and 6 physical cores
.NET Core SDK=5.0.100-rc.1.20413.9
  [Host]     : .NET Core 5.0.0 (CoreCLR 5.0.20.40416, CoreFX 5.0.20.40416), X64 RyuJIT
  Job-LOBVIX : .NET Core 5.0.0 (CoreCLR 5.0.20.40416, CoreFX 5.0.20.40416), X64 RyuJIT

PowerPlanMode=00000000-0000-0000-0000-000000000000  Runtime=.NET Core 5.0  Arguments=/p:DebugType=portable  
Toolchain=netcoreapp5.0  IterationTime=250.0000 ms  MaxIterationCount=20  
MinIterationCount=15  WarmupCount=1  

Method	input	value	comparisonType	Mean	Error	StdDev	Median	Min	Max	Code Size	Gen 0	Gen 1	Gen 2	Allocated
IndexOfString	AAAAA5AAAA	5	InvariantCulture	37.57 ns	0.708 ns	0.696 ns	37.54 ns	36.42 ns	38.80 ns	780 B	-	-	-	-
IndexOfString	AAAAAAAAAAAA(...)AAAAAAAAAAAA [1000]	X	Ordinal	29.27 ns	0.362 ns	0.303 ns	29.15 ns	28.92 ns	29.92 ns	780 B	-	-	-	-
IndexOfString	AAAAAAAAAAAA(...)AAAAAAAAAAAA [100]	x	InvariantCultureIgnoreCase	205.95 ns	5.799 ns	6.205 ns	204.95 ns	197.52 ns	220.59 ns	780 B	-	-	-	-
IndexOfString	AAAAAAAAAAAA(...)AAAAAAAAAAAA [100]	x	OrdinalIgnoreCase	575.32 ns	6.667 ns	5.205 ns	574.53 ns	569.91 ns	587.35 ns	780 B	-	-	-	-
IndexOfString	ABCDE	c	InvariantCultureIgnoreCase	32.66 ns	0.086 ns	0.077 ns	32.66 ns	32.53 ns	32.82 ns	780 B	-	-	-	-
IndexOfString	Hello Worldb(...)allylong!xyz [186]	w	OrdinalIgnoreCase	72.71 ns	1.117 ns	1.045 ns	72.47 ns	71.45 ns	74.58 ns	780 B	-	-	-	-
IndexOfString	Hello Worldb(...)allylong!xyz [187]	~	Ordinal	16.40 ns	0.130 ns	0.115 ns	16.35 ns	16.30 ns	16.63 ns	780 B	-	-	-	-
IndexOfString	Hello Worldbb(...)bbbbbbbbbbba! [47]	y	Ordinal	12.85 ns	0.092 ns	0.082 ns	12.82 ns	12.75 ns	13.02 ns	780 B	-	-	-	-
IndexOfString	More Test's	Tests	OrdinalIgnoreCase	82.42 ns	0.066 ns	0.055 ns	82.43 ns	82.33 ns	82.51 ns	780 B	-	-	-	-
IndexOfString	StrIng	string	OrdinalIgnoreCase	41.46 ns	0.158 ns	0.140 ns	41.43 ns	41.28 ns	41.74 ns	780 B	-	-	-	-
IndexOfString	foobardzsdzs	rddzs	InvariantCulture	45.39 ns	0.120 ns	0.101 ns	45.37 ns	45.29 ns	45.60 ns	780 B	-	-	-	-
IndexOfString	string1	string2	InvariantCulture	38.94 ns	0.092 ns	0.086 ns	38.92 ns	38.82 ns	39.08 ns	780 B	-	-	-	-
IndexOfString	だ	た	InvariantCulture	3,561.07 ns	11.114 ns	9.281 ns	3,563.64 ns	3,540.69 ns	3,572.37 ns	780 B	-	-	-	-
IndexOfString	だだだだだだだだだだだだ(...)だだだだだだだだだだだだ [100]	す	Ordinal	14.40 ns	0.245 ns	0.229 ns	14.57 ns	14.10 ns	14.60 ns	780 B	-	-	-	-
IndexOfString	だだだだだだだだだだだだ(...)だだだだだだだだだだだだ [1000]	x	Ordinal	29.33 ns	0.567 ns	0.502 ns	29.41 ns	28.65 ns	30.13 ns	780 B	-	-	-	-