[Perf] Changes at 5/18/2022 7:52:18 PM

[performanceautofiler]

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	a598ebb67973f03b1d6b3fd6c50f75bae3fe1c51
Compare	fbef29b711d94258ea872e15309366c369e36630
Diff	Diff

Regressions in System.Globalization.Tests.StringSearch

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
LastIndexOf_Word_NotFound - Duration of single invocation	45.07 μs	48.25 μs	1.07	0.14	False
LastIndexOf_Word_NotFound - Duration of single invocation	20.92 μs	24.36 μs	1.16	0.07	True
IndexOf_Word_NotFound - Duration of single invocation	20.90 μs	24.38 μs	1.17	0.17	False
LastIndexOf_Word_NotFound - Duration of single invocation	20.99 μs	25.11 μs	1.20	0.12	True
IndexOf_Word_NotFound - Duration of single invocation	22.45 μs	24.78 μs	1.10	0.12	True
LastIndexOf_Word_NotFound - Duration of single invocation	44.55 μs	48.75 μs	1.09	0.13	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreCase, True))

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.24600228452752 > 47.14012390804083.
IsChangePoint: Marked as a change because one of 5/18/2022 2:33:42 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 3:27:46 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -38.75166950973694 (T) = (0 -48009.148732002104) / Math.Sqrt((101200.88474754331 / (36)) + (149211.93231926844 / (33))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (33) - 2, .025) and -0.07425138728233723 = (44690.79519036658 - 48009.148732002104) / 44690.79519036658 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, IgnoreCase, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.36083597893915 > 22.087982811572772.
IsChangePoint: Marked as a change because one of 5/18/2022 2:33:42 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 3:27:46 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -19.90175482027051 (T) = (0 -24841.07946239808) / Math.Sqrt((235301.6757333725 / (36)) + (996440.1032600119 / (34))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (34) - 2, .025) and -0.1787985568413898 = (21073.218420762376 - 24841.07946239808) / 21073.218420762376 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (, IgnoreCase, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.379225260010404 > 21.963917017500297.
IsChangePoint: Marked as a change because one of 5/18/2022 10:04:15 AM, 5/24/2022 7:58:36 AM falls between 5/15/2022 3:27:46 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -28.360108142212283 (T) = (0 -24567.807405054442) / Math.Sqrt((94905.27229609009 / (34)) + (466458.3589744474 / (36))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (34) + (36) - 2, .025) and -0.16940496159410273 = (21008.810644658322 - 24567.807405054442) / 21008.810644658322 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 25.114084815835636 > 21.998996303190236.
IsChangePoint: Marked as a change because one of 5/18/2022 2:33:42 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 3:27:46 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -19.767148985777037 (T) = (0 -24890.468972114402) / Math.Sqrt((182635.64158130088 / (36)) + (1047952.534613519 / (34))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (34) - 2, .025) and -0.17711191636292012 = (21145.371672917736 - 24890.468972114402) / 21145.371672917736 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IndexOf_Word_NotFound(Options: (en-US, IgnoreCase, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 24.784446490885415 > 21.96632251129337.
IsChangePoint: Marked as a change because one of 5/18/2022 2:33:42 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 3:27:46 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -36.097947255042975 (T) = (0 -24472.698037088354) / Math.Sqrt((261826.87950130866 / (36)) + (45552.14964035206 / (34))) is less than -1.9954689314290597 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (34) - 2, .025) and -0.15860068258657556 = (21122.633884914572 - 24472.698037088354) / 21122.633884914572 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.LastIndexOf_Word_NotFound(Options: (, IgnoreCase, True))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 48.74863707165109 > 46.782092237357546.
IsChangePoint: Marked as a change because one of 5/18/2022 2:33:42 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 3:27:46 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -15.635535028777927 (T) = (0 -48449.90255132532) / Math.Sqrt((176590.99243474574 / (36)) + (1664237.424251793 / (33))) is less than -1.9960083540247138 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (33) - 2, .025) and -0.0821513366338729 = (44771.83635149684 - 48449.90255132532) / 44771.83635149684 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[lewing]

looks like this was probably dotnet/runtime#69318

cc @BrzVlad