[Perf] Changes at 4/9/2022 9:54:15 AM

[performanceautofiler]

Run Information

Architecture	x64
OS	alpine 3.12
Baseline	285b8259af30e255980b1ebfb2ddd60ca291a83a
Compare	0c64bdf3afc1fdc9166ad6c22c1990d3e0a6c509
Diff	Diff

Improvements in System.Globalization.Tests.StringSearch

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IndexOf_Word_NotFound - Duration of single invocation	442.61 ns	27.42 ns	0.06	0.02	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, OrdinalIgnoreCase, False))

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 27.42149016223144 < 420.6793749586912.
IsChangePoint: Marked as a change because one of 4/9/2022 5:43:54 AM, 4/12/2022 4:54:37 AM falls between 4/3/2022 1:52:36 PM and 4/12/2022 4:54:37 AM.
IsImprovementStdDev: Marked as improvement because 456.862079098238 (T) = (0 -27.438876512711303) / Math.Sqrt((249.20523626133115 / (299)) + (0.023278563086202672 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.9383413171154894 = (445.0123685597621 - 27.438876512711303) / 445.0123685597621 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	alpine 3.12
Baseline	285b8259af30e255980b1ebfb2ddd60ca291a83a
Compare	0c64bdf3afc1fdc9166ad6c22c1990d3e0a6c509
Diff	Diff

Improvements in System.Memory.ReadOnlySpan

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IndexOfString - Duration of single invocation	20.27 ns	14.88 ns	0.73	0.00	True
IndexOfString - Duration of single invocation	18.49 ns	9.09 ns	0.49	0.01	True
IndexOfString - Duration of single invocation	143.72 ns	11.98 ns	0.08	0.01	True
IndexOfString - Duration of single invocation	199.18 ns	180.47 ns	0.91	0.15	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "More Test's", value: "Tests", comparisonType: OrdinalIgnoreCase)

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 14.879510221248593 < 19.25449173472341.
IsChangePoint: Marked as a change because one of 4/9/2022 5:43:54 AM, 4/12/2022 4:54:37 AM falls between 4/3/2022 1:52:36 PM and 4/12/2022 4:54:37 AM.
IsImprovementStdDev: Marked as improvement because 134.6471289239642 (T) = (0 -14.451944549683928) / Math.Sqrt((0.011134322049222595 / (299)) + (0.021486776101157127 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.2848577151330936 = (20.208488374273028 - 14.451944549683928) / 20.208488374273028 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "w", comparisonType: OrdinalIgnoreCase)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 9.091534417323325 < 17.57158313198536.
IsChangePoint: Marked as a change because one of 4/9/2022 5:43:54 AM, 4/12/2022 4:54:37 AM falls between 4/3/2022 1:52:36 PM and 4/12/2022 4:54:37 AM.
IsImprovementStdDev: Marked as improvement because 121.87307605422795 (T) = (0 -9.166931930845704) / Math.Sqrt((0.011051236555621513 / (299)) + (0.06938624847502795 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.5035191374449302 = (18.463817283247057 - 9.166931930845704) / 18.463817283247057 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: OrdinalIgnoreCase)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 11.97687762504699 < 136.538257943585.
IsChangePoint: Marked as a change because one of 4/9/2022 5:43:54 AM, 4/12/2022 4:54:37 AM falls between 4/3/2022 1:52:36 PM and 4/12/2022 4:54:37 AM.
IsImprovementStdDev: Marked as improvement because 533.9595179588798 (T) = (0 -12.1356876077071) / Math.Sqrt((13.196866191016285 / (299)) + (0.2227224048086061 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.916785178377509 = (145.83565008120013 - 12.1356876077071) / 145.83565008120013 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Memory.ReadOnlySpan.IndexOfString(input: "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAXAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", value: "x", comparisonType: InvariantCultureIgnoreCase)

```log

Description of detection logic

IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed:Marked as improvement because 180.46595621388332 < 189.31035709707223.
IsChangePoint: Marked as a change because one of 2/10/2022 7:02:09 PM, 3/3/2022 10:59:13 PM, 3/17/2022 3:08:05 PM, 4/9/2022 5:43:54 AM, 4/12/2022 4:54:37 AM falls between 4/3/2022 1:52:36 PM and 4/12/2022 4:54:37 AM.
IsImprovementStdDev: Marked as improvement because 26.01439864066569 (T) = (0 -180.90461698604122) / Math.Sqrt((68.83830583547902 / (299)) + (0.824312323384889 / (12))) is greater than 1.9676708853560316 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (12) - 2, .975) and 0.07289089801313174 = (195.1276463561282 - 180.90461698604122) / 195.1276463561282 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[AndyAyersMS]

Looks like dotnet/runtime#67758