Regressions due to QJFL and OSR

[performanceautofiler]

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare	43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff	Diff

Regressions in System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
FixedTimeEquals_256Bit_Equal - Duration of single invocation	89.28 ns	102.18 ns	1.14	0.19	False
FixedTimeEquals_256Bit_FirstBitDifferent - Duration of single invocation	89.22 ns	102.15 ns	1.14	0.19	False
FixedTimeEquals_256Bit_LastBitDifferent - Duration of single invocation	89.24 ns	102.32 ns	1.15	0.17	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals*'

Payloads

Baseline
Compare

Histogram

System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_Equal

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 102.18106875445687 > 93.67357576127858.
IsChangePoint: Marked as a change because one of 2/1/2022 10:36:16 AM, 2/10/2022 10:02:40 PM, 2/19/2022 11:51:48 PM, 3/2/2022 5:56:50 PM, 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -26.645123210765615 (T) = (0 -102.26466856880218) / Math.Sqrt((6.972934543997907 / (32)) + (0.014138234168107212 / (23))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (23) - 2, .025) and -0.13868890519403115 = (89.80913759880396 - 102.26466856880218) / 89.80913759880396 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_FirstBitDifferent

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 102.14583971078534 > 93.74595748297735.
IsChangePoint: Marked as a change because one of 2/1/2022 8:13:55 AM, 2/10/2022 10:02:40 PM, 2/19/2022 11:51:48 PM, 3/2/2022 10:01:16 PM, 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -8.794442273741547 (T) = (0 -103.89171194128237) / Math.Sqrt((0.5633520995001862 / (32)) + (61.61518226641661 / (23))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (23) - 2, .025) and -0.16144661169133803 = (89.45026908296002 - 103.89171194128237) / 89.45026908296002 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_LastBitDifferent

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 102.31528334885473 > 93.7021903316295.
IsChangePoint: Marked as a change because one of 2/1/2022 1:24:52 PM, 2/10/2022 10:02:40 PM, 2/19/2022 11:51:48 PM, 3/3/2022 10:59:13 PM, 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -80.60357227108018 (T) = (0 -102.37083845998689) / Math.Sqrt((0.4294369988011269 / (32)) + (0.28554969333167635 / (23))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (23) - 2, .025) and -0.14489326151940368 = (89.41518122320777 - 102.37083845998689) / 89.41518122320777 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare	43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff	Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SingleWithPredicate_FirstElementMatches - Duration of single invocation	700.53 ns	748.74 ns	1.07	0.17	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: List)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 748.7365295592631 > 735.5346141759802.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -132.90361373287402 (T) = (0 -747.4129623501557) / Math.Sqrt((0.6017217235902365 / (32)) + (2.465665664171783 / (23))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (23) - 2, .025) and -0.06737353632595831 = (700.2356128510086 - 747.4129623501557) / 700.2356128510086 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare	43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff	Diff

Regressions in System.Collections.CopyTo<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Memory - Duration of single invocation	113.10 ns	186.25 ns	1.65	0.47	True
Array - Duration of single invocation	104.13 ns	189.38 ns	1.82	0.46	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CopyTo<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CopyTo<Int32>.Memory(Size: 2048)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 186.25349540239822 > 114.31609677281067.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -52.401497672779875 (T) = (0 -186.39345735134467) / Math.Sqrt((39.80100184985866 / (32)) + (21.273142535147162 / (23))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (23) - 2, .025) and -0.7065193128209458 = (109.22434686263743 - 186.39345735134467) / 109.22434686263743 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Collections.CopyTo<Int32>.Array(Size: 2048)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 189.37599091260634 > 108.87621133172183.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -50.6212873525026 (T) = (0 -186.07742709730286) / Math.Sqrt((32.25142518999982 / (32)) + (30.699100172727753 / (23))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (23) - 2, .025) and -0.7134401612374727 = (108.5987309664289 - 186.07742709730286) / 108.5987309664289 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare	43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff	Diff

Regressions in Benchstone.MDBenchI.MDMidpoint

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	537.18 ms	564.96 ms	1.05	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchI.MDMidpoint*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchI.MDMidpoint.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 564.9573121428572 > 564.0104052025.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -6.1740918192193295 (T) = (0 -568420734.6516166) / Math.Sqrt((313549356908744.56 / (32)) + (251211247070160.97 / (23))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (23) - 2, .025) and -0.05201476594206948 = (540316308.3386962 - 568420734.6516166) / 540316308.3386962 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare	43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff	Diff

Regressions in PerfLabTests.DictionaryExpansion

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ExpandDictionaries - Duration of single invocation	482.08 ns	529.82 ns	1.10	0.04	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.DictionaryExpansion*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.DictionaryExpansion.ExpandDictionaries

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 529.8239722937471 > 518.0439517892836.
IsChangePoint: Marked as a change because one of 3/9/2022 9:14:37 PM, 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -24.732588419526966 (T) = (0 -537.3535312682171) / Math.Sqrt((75.17297280022737 / (32)) + (32.62400690946964 / (23))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (23) - 2, .025) and -0.09810352051451968 = (489.34687962428035 - 537.3535312682171) / 489.34687962428035 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare	43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff	Diff

Regressions in System.Collections.IndexerSetReverse<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IList - Duration of single invocation	899.22 ns	1.11 μs	1.24	0.29	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSetReverse<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSetReverse<Int32>.IList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1112840347179558 > 944.1094567083051.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -21.797451002419997 (T) = (0 -1108.0169855982924) / Math.Sqrt((2181.5163529691968 / (32)) + (60.37187752466307 / (23))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (23) - 2, .025) and -0.19836037867642406 = (924.6108310273785 - 1108.0169855982924) / 924.6108310273785 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare	43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff	Diff

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
HashSet - Duration of single invocation	2.19 μs	3.04 μs	1.39	0.26	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<Int32>.HashSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.0393642748607435 > 2.2991612020932917.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/21/2022 1:28:14 PM, 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -68.39684303987657 (T) = (0 -3061.3422679012465) / Math.Sqrt((4843.692159791088 / (32)) + (244.3702559013411 / (23))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (23) - 2, .025) and -0.3973506217664565 = (2190.818982877225 - 3061.3422679012465) / 2190.818982877225 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare	43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff	Diff

Regressions in BenchmarksGame.BinaryTrees_2

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
RunBench - Duration of single invocation	108.58 ms	130.86 ms	1.21	0.07	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'BenchmarksGame.BinaryTrees_2*'

Payloads

Baseline
Compare

Histogram

BenchmarksGame.BinaryTrees_2.RunBench

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 130.8609133888889 > 113.92479392000001.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -11.11904022002931 (T) = (0 -128609086.14732479) / Math.Sqrt((1889710204573.7827 / (32)) + (81853945566789.62 / (23))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (23) - 2, .025) and -0.19681174376418314 = (107459746.1275126 - 128609086.14732479) / 107459746.1275126 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

category:performance
theme:osr

[EgorBo]

#65675

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[EgorBo]

More:
dotnet/perf-autofiling-issues#4369,
dotnet/perf-autofiling-issues#4369
dotnet/perf-autofiling-issues#4384
dotnet/perf-autofiling-issues#4386
dotnet/perf-autofiling-issues#4387
dotnet/perf-autofiling-issues#4392
dotnet/perf-autofiling-issues#4393
dotnet/perf-autofiling-issues#4396

[AndyAyersMS]

Related comparison report

https://pvscmdupload.blob.core.windows.net/drewtest/reports/report_Weekly_ca%3Dx64_cb%3Drefs-heads-main_co%3DWindows1018362_cr%3Ddotnetruntime_cc%3DCompilationMode%3Dtiered-PGOType%3Dnopgo-RunKind%3Dmicro_Baseline_bc%3DCompilationMode%3Dtiered-RunKind%3Dmicro_2022-04-05.html

[AndyAyersMS]

There are generally two root causes for these:

• benchmark is long running and measures few iterations, and so is measuring OSR codegen. In some cases performance of OSR will be less than Tier1:
  • lack of loop alignment
  • lack of loop cloning
  • lack of struct promotion
• benchmark measures Tier1 code, but running unoptimized code initially alters memory alignment

Initial work for this issue is to categorize the regressions using or extending the above. Will probably wait a few days before digging in so we also see deferred impact and arm64 data.

[ghost]

Tagging subscribers to this area: @JulieLeeMSFT
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare	43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff	Diff

Regressions in System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
FixedTimeEquals_256Bit_Equal - Duration of single invocation	89.28 ns	102.18 ns	1.14	0.19	False
FixedTimeEquals_256Bit_FirstBitDifferent - Duration of single invocation	89.22 ns	102.15 ns	1.14	0.19	False
FixedTimeEquals_256Bit_LastBitDifferent - Duration of single invocation	89.24 ns	102.32 ns	1.15	0.17	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals*'

Payloads

Baseline
Compare

Histogram

System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_Equal

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 102.18106875445687 > 93.67357576127858.
IsChangePoint: Marked as a change because one of 2/1/2022 10:36:16 AM, 2/10/2022 10:02:40 PM, 2/19/2022 11:51:48 PM, 3/2/2022 5:56:50 PM, 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -26.645123210765615 (T) = (0 -102.26466856880218) / Math.Sqrt((6.972934543997907 / (32)) + (0.014138234168107212 / (23))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (23) - 2, .025) and -0.13868890519403115 = (89.80913759880396 - 102.26466856880218) / 89.80913759880396 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_FirstBitDifferent

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 102.14583971078534 > 93.74595748297735.
IsChangePoint: Marked as a change because one of 2/1/2022 8:13:55 AM, 2/10/2022 10:02:40 PM, 2/19/2022 11:51:48 PM, 3/2/2022 10:01:16 PM, 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -8.794442273741547 (T) = (0 -103.89171194128237) / Math.Sqrt((0.5633520995001862 / (32)) + (61.61518226641661 / (23))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (23) - 2, .025) and -0.16144661169133803 = (89.45026908296002 - 103.89171194128237) / 89.45026908296002 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_LastBitDifferent

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 102.31528334885473 > 93.7021903316295.
IsChangePoint: Marked as a change because one of 2/1/2022 1:24:52 PM, 2/10/2022 10:02:40 PM, 2/19/2022 11:51:48 PM, 3/3/2022 10:59:13 PM, 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -80.60357227108018 (T) = (0 -102.37083845998689) / Math.Sqrt((0.4294369988011269 / (32)) + (0.28554969333167635 / (23))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (23) - 2, .025) and -0.14489326151940368 = (89.41518122320777 - 102.37083845998689) / 89.41518122320777 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare	43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff	Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
SingleWithPredicate_FirstElementMatches - Duration of single invocation	700.53 ns	748.74 ns	1.07	0.17	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: List)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 748.7365295592631 > 735.5346141759802.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -132.90361373287402 (T) = (0 -747.4129623501557) / Math.Sqrt((0.6017217235902365 / (32)) + (2.465665664171783 / (23))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (23) - 2, .025) and -0.06737353632595831 = (700.2356128510086 - 747.4129623501557) / 700.2356128510086 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare	43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff	Diff

Regressions in System.Collections.CopyTo<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Memory - Duration of single invocation	113.10 ns	186.25 ns	1.65	0.47	True
Array - Duration of single invocation	104.13 ns	189.38 ns	1.82	0.46	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.CopyTo<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.CopyTo<Int32>.Memory(Size: 2048)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 186.25349540239822 > 114.31609677281067.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -52.401497672779875 (T) = (0 -186.39345735134467) / Math.Sqrt((39.80100184985866 / (32)) + (21.273142535147162 / (23))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (23) - 2, .025) and -0.7065193128209458 = (109.22434686263743 - 186.39345735134467) / 109.22434686263743 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Collections.CopyTo<Int32>.Array(Size: 2048)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 189.37599091260634 > 108.87621133172183.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -50.6212873525026 (T) = (0 -186.07742709730286) / Math.Sqrt((32.25142518999982 / (32)) + (30.699100172727753 / (23))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (23) - 2, .025) and -0.7134401612374727 = (108.5987309664289 - 186.07742709730286) / 108.5987309664289 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare	43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff	Diff

Regressions in Benchstone.MDBenchI.MDMidpoint

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Test - Duration of single invocation	537.18 ms	564.96 ms	1.05	0.02	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.MDBenchI.MDMidpoint*'

Payloads

Baseline
Compare

Histogram

Benchstone.MDBenchI.MDMidpoint.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 564.9573121428572 > 564.0104052025.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -6.1740918192193295 (T) = (0 -568420734.6516166) / Math.Sqrt((313549356908744.56 / (32)) + (251211247070160.97 / (23))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (23) - 2, .025) and -0.05201476594206948 = (540316308.3386962 - 568420734.6516166) / 540316308.3386962 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare	43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff	Diff

Regressions in PerfLabTests.DictionaryExpansion

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ExpandDictionaries - Duration of single invocation	482.08 ns	529.82 ns	1.10	0.04	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.DictionaryExpansion*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.DictionaryExpansion.ExpandDictionaries

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 529.8239722937471 > 518.0439517892836.
IsChangePoint: Marked as a change because one of 3/9/2022 9:14:37 PM, 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -24.732588419526966 (T) = (0 -537.3535312682171) / Math.Sqrt((75.17297280022737 / (32)) + (32.62400690946964 / (23))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (23) - 2, .025) and -0.09810352051451968 = (489.34687962428035 - 537.3535312682171) / 489.34687962428035 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare	43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff	Diff

Regressions in System.Collections.IndexerSetReverse<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
IList - Duration of single invocation	899.22 ns	1.11 μs	1.24	0.29	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSetReverse<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSetReverse<Int32>.IList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1112840347179558 > 944.1094567083051.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -21.797451002419997 (T) = (0 -1108.0169855982924) / Math.Sqrt((2181.5163529691968 / (32)) + (60.37187752466307 / (23))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (23) - 2, .025) and -0.19836037867642406 = (924.6108310273785 - 1108.0169855982924) / 924.6108310273785 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare	43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff	Diff

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
HashSet - Duration of single invocation	2.19 μs	3.04 μs	1.39	0.26	False

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue<Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<Int32>.HashSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.0393642748607435 > 2.2991612020932917.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/21/2022 1:28:14 PM, 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -68.39684303987657 (T) = (0 -3061.3422679012465) / Math.Sqrt((4843.692159791088 / (32)) + (244.3702559013411 / (23))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (23) - 2, .025) and -0.3973506217664565 = (2190.818982877225 - 3061.3422679012465) / 2190.818982877225 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture	x64
OS	ubuntu 18.04
Baseline	5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare	43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff	Diff

Regressions in BenchmarksGame.BinaryTrees_2

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
RunBench - Duration of single invocation	108.58 ms	130.86 ms	1.21	0.07	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'BenchmarksGame.BinaryTrees_2*'

Payloads

Baseline
Compare

Histogram

BenchmarksGame.BinaryTrees_2.RunBench

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 130.8609133888889 > 113.92479392000001.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/5/2022 12:43:03 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 12:43:03 AM.
IsRegressionStdDev: Marked as regression because -11.11904022002931 (T) = (0 -128609086.14732479) / Math.Sqrt((1889710204573.7827 / (32)) + (81853945566789.62 / (23))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (23) - 2, .025) and -0.19681174376418314 = (107459746.1275126 - 128609086.14732479) / 107459746.1275126 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	performanceautofiler[bot]
Assignees:	AndyAyersMS
Labels:	area-CodeGen-coreclr, untriaged, refs/heads/main, ubuntu 18.04, RunKind=micro, PGO, Regression, CoreClr, x64
Milestone:	7.0.0

[EgorBo]

Arm64 dotnet/perf-autofiling-issues#4424, dotnet/perf-autofiling-issues#4415, dotnet/perf-autofiling-issues#4414

[dakersnar]

As @AndyAyersMS and I discussed today, the following regressions from the perf report are likely caused by OSR.

Benchstone.BenchI.Array2.Test

Result	Ratio	Alloc Delta	Operating System	Bit	Processor Name
Same	0.97	+0	ubuntu 20.04	Arm64	Unknown processor
Slower	0.65	+0	Windows 10	Arm64	Microsoft SQ1 3.0 GHz
Same	1.05	+0	Windows 11	Arm64	Microsoft SQ1 3.0 GHz
Slower	0.86	+0	Windows 11	Arm64	Unknown processor
Slower	0.66	-792	macOS Monterey 12.3	Arm64	Apple M1 Max
Slower	0.70	+0	Windows 10	X64	Intel Xeon Platinum 8272CL CPU 2.60GHz
Slower	0.67	-144	Windows 10	X64	Intel Xeon CPU E5-1650 v4 3.60GHz
Slower	0.67	-3176	Windows 10	X64	Intel Core i7-6700 CPU 3.40GHz (Skylake)
Slower	0.63	+0	Windows 11	X64	AMD Ryzen Threadripper PRO 3945WX 12-Cores
Slower	0.68	+0	Windows 11	X64	AMD Ryzen 9 5950X
Slower	0.69	+0	Windows 11	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Slower	0.73	+0	Windows 11	X64	Intel Core i9-9900T CPU 2.10GHz
Slower	0.67	+0	alpine 3.13	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Slower	0.48	+0	pop 22.04	X64	Intel Core i7-6600U CPU 2.60GHz (Skylake)
Slower	0.63	+0	ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz
Slower	0.65	-4312	ubuntu 18.04	X64	Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)
Slower	0.63	-792	ubuntu 20.04	X64	AMD Ryzen 9 5900X
Slower	0.54	+0	ubuntu 20.04	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Same	1.00	+0	Windows 10	X86	Intel Xeon CPU E5-1650 v4 3.60GHz
Same	0.95	+0	Windows 10	X86	Intel Core i7-6700 CPU 3.40GHz (Skylake)
Same	1.00	-356	Windows 11	X86	AMD Ryzen Threadripper PRO 3945WX 12-Cores
Slower	0.62	-792	macOS Monterey 12.2.1	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)
Slower	0.65	+0	macOS Monterey 12.3.1	X64	Intel Core i7-4870HQ CPU 2.50GHz (Haswell)

SciMark2.kernel.benchSparseMult

Result	Ratio	Alloc Delta	Operating System	Bit	Processor Name
Same	0.94	+0	ubuntu 20.04	Arm64	Unknown processor
Same	0.94	+0	Windows 10	Arm64	Microsoft SQ1 3.0 GHz
Same	0.94	-792	Windows 11	Arm64	Microsoft SQ1 3.0 GHz
Same	0.94	+0	Windows 11	Arm64	Unknown processor
Same	0.92	+0	macOS Monterey 12.3	Arm64	Apple M1 Max
Slower	0.88	+0	Windows 10	X64	Intel Xeon Platinum 8272CL CPU 2.60GHz
Same	0.99	-840	Windows 10	X64	Intel Xeon CPU E5-1650 v4 3.60GHz
Slower	0.88	-144	Windows 10	X64	Intel Core i7-6700 CPU 3.40GHz (Skylake)
Same	0.92	-792	Windows 11	X64	AMD Ryzen Threadripper PRO 3945WX 12-Cores
Same	0.94	-336	Windows 11	X64	AMD Ryzen 9 5950X
Slower	0.88	+0	Windows 11	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Slower	0.88	+0	Windows 11	X64	Intel Core i9-9900T CPU 2.10GHz
Same	0.92	+0	alpine 3.13	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Slower	0.87	-792	pop 22.04	X64	Intel Core i7-6600U CPU 2.60GHz (Skylake)
Slower	0.49	+0	ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz
Same	0.93	+0	ubuntu 18.04	X64	Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)
Same	0.98	+0	ubuntu 20.04	X64	AMD Ryzen 9 5900X
Slower	0.81	+0	ubuntu 20.04	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Same	1.00	+0	Windows 10	X86	Intel Xeon CPU E5-1650 v4 3.60GHz
Same	0.98	-1416	Windows 10	X86	Intel Core i7-6700 CPU 3.40GHz (Skylake)
Same	1.00	+0	Windows 11	X86	AMD Ryzen Threadripper PRO 3945WX 12-Cores
Slower	0.51	+0	macOS Monterey 12.2.1	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)
Slower	0.49	+0	macOS Monterey 12.3.1	X64	Intel Core i7-4870HQ CPU 2.50GHz (Haswell)

System.Collections.CopyTo.ReadOnlySpan(Size: 2048)

Result	Ratio	Alloc Delta	Operating System	Bit	Processor Name
Slower	0.87	+0	ubuntu 20.04	Arm64	Unknown processor
Slower	0.89	+0	Windows 10	Arm64	Microsoft SQ1 3.0 GHz
Same	0.98	+0	Windows 11	Arm64	Microsoft SQ1 3.0 GHz
Slower	0.89	+0	Windows 11	Arm64	Unknown processor
Slower	0.89	+0	macOS Monterey 12.3	Arm64	Apple M1 Max
Slower	0.68	+0	Windows 10	X64	Intel Xeon Platinum 8272CL CPU 2.60GHz
Same	1.03	+0	Windows 10	X64	Intel Xeon CPU E5-1650 v4 3.60GHz
Slower	0.60	+0	Windows 10	X64	Intel Core i7-6700 CPU 3.40GHz (Skylake)
Same	0.99	+0	Windows 11	X64	AMD Ryzen Threadripper PRO 3945WX 12-Cores
Same	1.01	+0	Windows 11	X64	AMD Ryzen 9 5950X
Slower	0.59	+0	Windows 11	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Slower	0.61	+0	Windows 11	X64	Intel Core i9-9900T CPU 2.10GHz
Slower	0.59	+0	alpine 3.13	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Faster	1.65	+0	pop 22.04	X64	Intel Core i7-6600U CPU 2.60GHz (Skylake)
Slower	0.80	+0	ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz
Slower	0.71	+0	ubuntu 18.04	X64	Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)
Same	1.02	+0	ubuntu 20.04	X64	AMD Ryzen 9 5900X
Slower	0.64	+0	ubuntu 20.04	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Slower	0.68	+0	Windows 10	X86	Intel Xeon CPU E5-1650 v4 3.60GHz
Same	1.02	+0	Windows 10	X86	Intel Core i7-6700 CPU 3.40GHz (Skylake)
Same	0.99	+0	Windows 11	X86	AMD Ryzen Threadripper PRO 3945WX 12-Cores
Same	0.99	+0	macOS Monterey 12.2.1	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)
Same	0.99	+0	macOS Monterey 12.3.1	X64	Intel Core i7-4870HQ CPU 2.50GHz (Haswell)

LinqBenchmarks.Count00ForX

Result	Ratio	Alloc Delta	Operating System	Bit	Processor Name	Modality
Slower	0.66	+408	ubuntu 20.04	Arm64	Unknown processor
Slower	0.69	+240	Windows 10	Arm64	Microsoft SQ1 3.0 GHz
Slower	0.63	+240	Windows 11	Arm64	Microsoft SQ1 3.0 GHz
Slower	0.69	+240	Windows 11	Arm64	Unknown processor
Same	1.00	-256	macOS Monterey 12.3	Arm64	Apple M1 Max	several?
Slower	0.63	+240	Windows 10	X64	Intel Xeon Platinum 8272CL CPU 2.60GHz	several?
Same	0.59	+120	Windows 10	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	bimodal
Same	0.45	+120	Windows 10	X64	Intel Core i7-6700 CPU 3.40GHz (Skylake)	bimodal
Same	0.69	+120	Windows 11	X64	AMD Ryzen Threadripper PRO 3945WX 12-Cores	bimodal
Same	1.01	+0	Windows 11	X64	AMD Ryzen 9 5950X	several?
Same	0.72	-78	Windows 11	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)	bimodal
Same	0.72	+120	Windows 11	X64	Intel Core i9-9900T CPU 2.10GHz	bimodal
Same	0.47	+204	alpine 3.13	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)	bimodal
Slower	0.45	+612	pop 22.04	X64	Intel Core i7-6600U CPU 2.60GHz (Skylake)
Same	0.54	+204	ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	bimodal
Slower	0.65	+408	ubuntu 18.04	X64	Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)
Same	1.00	+0	ubuntu 20.04	X64	AMD Ryzen 9 5900X	several?
Same	0.54	+204	ubuntu 20.04	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)	bimodal
Same	0.99	+0	Windows 10	X86	Intel Xeon CPU E5-1650 v4 3.60GHz
Same	0.98	+0	Windows 10	X86	Intel Core i7-6700 CPU 3.40GHz (Skylake)
Same	0.98	+0	Windows 11	X86	AMD Ryzen Threadripper PRO 3945WX 12-Cores
Slower	0.54	+408	macOS Monterey 12.2.1	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)
Slower	0.53	+612	macOS Monterey 12.3.1	X64	Intel Core i7-4870HQ CPU 2.50GHz (Haswell)

System.Collections.CopyTo.Memory(Size: 2048)

Result	Ratio	Operating System	Bit	Processor Name
Slower	0.88	ubuntu 20.04	Arm64	Unknown processor
Same	1.00	Windows 10	Arm64	Microsoft SQ1 3.0 GHz
Same	0.90	Windows 11	Arm64	Microsoft SQ1 3.0 GHz
Same	1.00	Windows 11	Arm64	Unknown processor
Slower	0.65	macOS Monterey 12.3	Arm64	Apple M1 Max
Slower	0.69	Windows 10	X64	Intel Xeon Platinum 8272CL CPU 2.60GHz
Same	1.01	Windows 10	X64	Intel Xeon CPU E5-1650 v4 3.60GHz
Slower	0.61	Windows 10	X64	Intel Core i7-6700 CPU 3.40GHz (Skylake)
Same	1.00	Windows 11	X64	AMD Ryzen Threadripper PRO 3945WX 12-Cores
Same	1.01	Windows 11	X64	AMD Ryzen 9 5950X
Slower	0.59	Windows 11	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Slower	0.62	Windows 11	X64	Intel Core i9-9900T CPU 2.10GHz
Slower	0.56	alpine 3.13	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Faster	1.56	pop 22.04	X64	Intel Core i7-6600U CPU 2.60GHz (Skylake)
Slower	0.71	ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz
Slower	0.70	ubuntu 18.04	X64	Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)
Faster	1.14	ubuntu 20.04	X64	AMD Ryzen 9 5900X
Slower	0.65	ubuntu 20.04	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Slower	0.56	Windows 10	X86	Intel Xeon CPU E5-1650 v4 3.60GHz
Same	0.98	Windows 10	X86	Intel Core i7-6700 CPU 3.40GHz (Skylake)
Same	1.00	Windows 11	X86	AMD Ryzen Threadripper PRO 3945WX 12-Cores
Same	0.97	macOS Monterey 12.2.1	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)
Same	1.00	macOS Monterey 12.3.1	X64	Intel Core i7-4870HQ CPU 2.50GHz (Haswell)

[AndyAyersMS]

For FractalPerf, BDN chooses 1 invocation per iteration, so the benchmark never reaches Tier1 when OSR is enabled. So, we're measuring the perf of OSR code.

The main issue is that the complex structs don't get promoted in the OSR methods, and this impacts the inner loop codegen in both Mandlebrot:Render and Julia:Render.

For instance, in Mandelbrot:Render (below) the extra vmovsd and vmovaps are caused by access to accum. Note (interestingly) the uses of num get hoisted as it's not written.

A more general SROA approach might work nicely here since we know exactly what gets written and that the loop won't cause exceptions (so any store back to accum could be done on exit). Even without store sinking, in-loop stores might be cheap enough not to cause serious regressions (since nothing will depend on them).

;; OSR

G_M63708_IG06:              ;; offset=009EH
       C5FB10BC24B8000000   vmovsd   xmm7, qword ptr [rsp+B8H]
       C54359C7             vmulsd   xmm8, xmm7, xmm7
       C57B108C24C0000000   vmovsd   xmm9, qword ptr [rsp+C0H]
       C4413359D1           vmulsd   xmm10, xmm9, xmm9
       C4413B5CC2           vsubsd   xmm8, xmm8, xmm10
       C4417828D0           vmovaps  xmm10, xmm8
       C5C358FF             vaddsd   xmm7, xmm7, xmm7
       C4C14359F9           vmulsd   xmm7, xmm7, xmm9
       C57828CF             vmovaps  xmm9, xmm7
       C57B119424B8000000   vmovsd   qword ptr [rsp+B8H], xmm10
       C57B118C24C0000000   vmovsd   qword ptr [rsp+C0H], xmm9
       C57828CD             vmovaps  xmm9, xmm5
       C57828D6             vmovaps  xmm10, xmm6
       C4413B58C1           vaddsd   xmm8, xmm8, xmm9
       C4417828C8           vmovaps  xmm9, xmm8
       C4C14358FA           vaddsd   xmm7, xmm7, xmm10
       C57828D7             vmovaps  xmm10, xmm7
       C57B118C24B8000000   vmovsd   qword ptr [rsp+B8H], xmm9
       C57B119424C0000000   vmovsd   qword ptr [rsp+C0H], xmm10
       C4413B59C0           vmulsd   xmm8, xmm8, xmm8
       C5C359FF             vmulsd   xmm7, xmm7, xmm7
       C5BB58FF             vaddsd   xmm7, xmm8, xmm7
       C5F92EFA             vucomisd xmm7, xmm2
       770D                 ja       SHORT G_M63708_IG07
       FFC0                 inc      eax
       3DE8030000           cmp      eax, 0x3E8
       0F8C6FFFFFFF         jl       G_M63708_IG06

;; Tier1

G_M63708_IG05:              ;; offset=0064H
       C4413B59D0           vmulsd   xmm10, xmm8, xmm8
       C4413359D9           vmulsd   xmm11, xmm9, xmm9
       C4412B5CD3           vsubsd   xmm10, xmm10, xmm11
       C4413B58C0           vaddsd   xmm8, xmm8, xmm8
       C4413B59C9           vmulsd   xmm9, xmm8, xmm9
       C52B58C6             vaddsd   xmm8, xmm10, xmm6
       C53358CF             vaddsd   xmm9, xmm9, xmm7
       C4413B59D0           vmulsd   xmm10, xmm8, xmm8
       C4413359D9           vmulsd   xmm11, xmm9, xmm9
       C4412B58D3           vaddsd   xmm10, xmm10, xmm11
       C5792ED1             vucomisd xmm10, xmm1
       7709                 ja       SHORT G_M63708_IG06
       FFC0                 inc      eax
       3DE8030000           cmp      eax, 0x3E8
       7CC1                 jl       SHORT G_M63708_IG05

[AndyAyersMS]

cc @jakobbotsch for the example just above.

OSR might provide a wealth of cases where more flexible promotion helps, since it is (overly) pessimistic about struct address exposure.

[AndyAyersMS]

We have known for a while now that OSR and Tier1/FullOpt perf were not always going to match. Seems like the number of places where this doesn't happen is acceptably small.

We will keep working on this in 8.0 to try and close the gap further. So am going to re-label this as an 8.0 issue.

[Jozkee]

@AndyAyersMS this regression was detected in the preview 4 report, but was never officially resolved, and showed up again on our RC2 vs 6 report. Is this still under investigation, or do we consider it to be by design?

FractalPerf.Launch.Test

Result	Ratio	Alloc Delta	Operating System	Bit	Processor Name	Modality
Slower	0.57	-1520	ubuntu 18.04	Arm64	Unknown processor
Slower	0.49	+220	Windows 11	Arm64	Unknown processor
Slower	0.54	-48	Windows 11	Arm64	Microsoft SQ1 3.0 GHz	several?
Slower	0.53	+240	Windows 11	Arm64	Microsoft SQ1 3.0 GHz
Slower	0.53	+84	macOS Monterey 12.6	Arm64	Apple M1
Slower	0.53	+80	macOS Monterey 12.6	Arm64	Apple M1 Max
Slower	0.78	-3096	Windows 10	X64	Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)
Slower	0.58	+240	Windows 11	X64	AMD Ryzen Threadripper PRO 3945WX 12-Cores
Slower	0.50	+360	Windows 11	X64	AMD Ryzen 9 5900X
Same	0.51	+120	Windows 11	X64	AMD Ryzen 9 7950X	bimodal
Slower	0.69	+192	Windows 11	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Slower	0.70	-552	debian 11	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Slower	0.53	+612	ubuntu 18.04	X64	AMD Ryzen 9 5900X
Slower	0.64	+408	ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz
Slower	0.51	+612	ubuntu 20.04	X64	AMD Ryzen 9 5900X
Faster	2.17	-4568	ubuntu 20.04	X64	Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)	bimodal
Slower	0.75	+408	ubuntu 20.04	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Slower	0.69	+0	macOS Big Sur 11.7	X64	Intel Core i5-4278U CPU 2.60GHz (Haswell)
Slower	0.70	+0	macOS Monterey 12.6	X64	Intel Core i7-4870HQ CPU 2.50GHz (Haswell)

[AndyAyersMS]

do we consider it to be by design?

Yes. There are a few tests where BDN's strategy and the length of time it takes to run one iteration causes us to measure OSR code, and where the OSR code doesn't match up with Tier1. You can see here BDN measures exactly one invocation which takes ~250ns or so.

[AndyAyersMS]

FWIW we may end up fixing some of these in .NET 8 by bringing the OSR code a bit closer to Tier1. See #76928.

[jakobbotsch]

cc @jakobbotsch for the example just above.

OSR might provide a wealth of cases where more flexible promotion helps, since it is (overly) pessimistic about struct address exposure.

FWIW we may end up fixing some of these in .NET 8 by bringing the OSR code a bit closer to Tier1. See #76928.

Are these cases in particular still relevant for physical promotion, or did #83910 mostly address these parity gaps?

[AndyAyersMS]

Using the handy new autofiling issue aggregator, here's where things stand wrt the issues linked here. Top number is current vs regression baseline, bottom is the reported regression (so if top number is less, things have improved since).

Only the top dozen or so are still showing regressions. Note we no longer run or triage alpine (there may be data for alpine 3.15).

Notes	Recent Score	Orig Score	x64-alpine-3.12	x64-ubuntu-18.04	x64-Windows-intel	x64-Windows-amd	Benchmark
fixed?	1.84	1.24				1.84 1.24	System.Tests.Perf_String.Trim(s: "Test ")
bimodal	1.63	1.65		1.63 1.65			System.Collections.CopyTo(Int32).Memory(Size: 2048)
bimodal obsolete?	1.56	1.26				1.56 1.26	System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\s+Holmes", Options: Compiled)
alpine	1.29	1.16	1.29 1.16				System.Globalization.Tests.StringEquality.Compare_Same_Upper(Count: 1024, Options: (en-US, OrdinalIgnoreCase))
notes	1.23	1.15				1.23 1.15	System.Tests.Perf_String.Trim_CharArr(s: "Test ", c: [' ', ' '])
alpine	1.17	1.28	1.49 1.48		0.92 1.10		Benchstone.BenchF.NewtR.Test
	1.14	1.38				1.14 1.38	System.Memory.Span(Byte).SequenceEqual(Size: 512)
	1.13	1.21		1.13 1.21			BenchmarksGame.BinaryTrees_2.RunBench
	1.10	1.16				1.10 1.16	System.Tests.Perf_String.Substring_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 0, i2: 8)
	1.09	1.09	1.06 1.10		1.13 1.09		Benchstone.BenchI.IniArray.Test
	1.09	1.10	1.11 1.11			1.07 1.10	SciMark2.kernel.benchSparseMult
	1.08	1.33				1.08 1.33	PerfLabTests.BlockCopyPerf.CallBlockCopy(numElements: 100)
	1.08	1.09			1.06 1.10	1.10 1.09	Benchstone.BenchF.Simpsn.Test
	1.08	1.08	1.08 1.08				PerfLabTests.LowLevelPerf.InterfaceInterfaceMethodSwitchCallType
	1.08	1.74	1.06 1.79	1.06 1.82	1.11 1.63		System.Collections.CopyTo(Int32).Array(Size: 2048)
	1.08	1.13				1.08 1.13	Microsoft.Extensions.Primitives.StringSegmentBenchmark.TrimEnd
	1.07	1.15	1.07 1.15				System.Collections.ContainsTrue(String).List(Size: 512)
	1.06	1.05	1.06 1.05				System.Collections.Sort(IntStruct).Array_ComparerStruct(Size: 512)
	1.06	1.07	1.06 1.07				System.Text.Json.Document.Tests.Perf_EnumerateArray.EnumerateUsingIndexer(TestCase: ArrayOfNumbers)
	1.03	1.10		1.03 1.10			PerfLabTests.DictionaryExpansion.ExpandDictionaries
	1.03	1.14				1.03 1.14	Microsoft.Extensions.Primitives.StringSegmentBenchmark.TrimStart
	1.03	1.34	1.03 1.34				System.Globalization.Tests.StringEquality.Compare_Same(Count: 1024, Options: (en-US, Ordinal))
	1.01	1.15		1.01 1.15			System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_LastBitDifferent
	1.01	1.14		1.01 1.14			System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_FirstBitDifferent
	1.00	1.14		1.00 1.14			System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_Equal
	1.00	1.16				1.00 1.16	System.Numerics.Tests.Perf_BigInteger.Divide(arguments: 16,8 bits)
	0.99	1.18	0.99 1.18				System.Linq.Tests.Perf_Enumerable.WhereFirst_LastElementMatches(input: List)
	0.96	1.13				0.96 1.13	System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,no (escaping /) required,16)
	0.96	1.27				0.96 1.27	System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt16(value: 32767)
	0.95	1.11				0.95 1.11	System.Tests.Perf_UInt16.ToString(value: 12345)
	0.95	1.15				0.95 1.15	System.Numerics.Tests.Perf_VectorOf(UInt64).GetHashCodeBenchmark
	0.95	1.23				0.95 1.23	System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 2, Options: Compiled)
	0.95	1.07				0.95 1.07	System.Tests.Perf_Int64.ParseSpan(value: "-9223372036854775808")
	0.95	1.14				0.95 1.14	System.Numerics.Tests.Perf_VectorOf(Int64).GetHashCodeBenchmark
	0.95	1.30				0.95 1.30	System.Collections.Tests.Perf_BitArray.BitArrayRightShift(Size: 4)
	0.95	1.39		0.95 1.39			System.Collections.ContainsTrue(Int32).HashSet(Size: 512)
	0.94	1.06				0.94 1.06	System.Tests.Perf_String.GetChars(size: 100)
	0.93	1.15				0.93 1.15	System.Tests.Perf_UInt16.Parse(value: "65535")
	0.91	1.10				0.91 1.10	System.Text.Tests.Perf_Encoding.GetBytes(size: 512, encName: "utf-8")
	0.89	1.33				0.89 1.33	System.Tests.Perf_Int32.ParseSpan(value: "12345")
	0.88	1.22				0.88 1.22	System.Tests.Perf_Int32.TryParse(value: "12345")
	0.86	1.17				0.86 1.17	System.Tests.Perf_Int32.TryParseSpan(value: "-2147483648")
	0.86	1.30	0.86 1.30				Microsoft.Extensions.Primitives.StringSegmentBenchmark.LastIndexOf
	0.86	1.20				0.86 1.20	System.Tests.Perf_Int16.Parse(value: "0")
	0.85	1.17				0.85 1.17	System.Tests.Perf_UInt32.Parse(value: "12345")
	0.83	1.17				0.83 1.17	System.Tests.Perf_UInt16.Parse(value: "0")
	0.82	1.11				0.82 1.11	System.Tests.Perf_Int32.ToStringHex(value: -2147483648)
	0.76	1.11				0.76 1.11	System.Tests.Perf_UInt32.TryParse(value: "0")
	0.74	1.14	1.07 1.18			0.52 1.11	System.Collections.IterateForEachNonGeneric(Int32).Queue(Size: 512)
	0.72	1.16				0.72 1.16	System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: HelloWorld)
	0.72	1.08				0.72 1.08	System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: -1.7976931348623157E+308)
	0.70	1.15				0.70 1.15	System.MathBenchmarks.Double.ScaleB
	0.67	1.20				0.67 1.20	System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input: IList)
	0.66	1.05		0.66 1.05			Benchstone.MDBenchI.MDMidpoint.Test
	0.62	1.24		0.62 1.24			System.Collections.IndexerSetReverse(Int32).IList(Size: 512)
	0.59	1.09			0.59 1.09		System.Collections.TryGetValueFalse(Int32, Int32).ImmutableSortedDictionary(Size: 512)
	0.50	1.12				0.50 1.12	System.Globalization.Tests.StringEquality.Compare_DifferentFirstChar(Count: 1024, Options: (en-US, Ordinal))
	0.48	1.14			0.48 1.14		System.Linq.Tests.Perf_Enumerable.Max(input: IEnumerable)
	0.34	1.07		0.34 1.07			System.Linq.Tests.Perf_Enumerable.SingleWithPredicate_FirstElementMatches(input: List)
	0.28	1.07			0.28 1.07		System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 0, 1024])

[AndyAyersMS]

System.Tests.Perf_String.Trim(s: "Test ")

Table above has some stale global history entries, and this now seems to be finally at par (OSR was enabled at the end of March 2022):

[AndyAyersMS]

System.Collections.CopyTo<Int32>.Memory(Size: 2048)

Became bimodal once OSR was enabled, but doesn't really look like it regressed.

[AndyAyersMS]

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_RustLang_Sherlock.Count(Pattern: "Sherlock\s+Holmes", Options: Compiled)

Seems like we stopped running this in Oct 2022?

[AndyAyersMS]

System.Tests.Perf_String.Trim_CharArr(s: "Test ", c: [' ', ' '])

Possibly still regressed a bit, reference level is around 13.6ns. For some reason I cannot get this to latch via ADE query. But regression would be pretty small.

[AndyAyersMS]

I think most of these are since mitigated. So will close and we can re-open specific issues later if necessary.