Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Changes at 5/11/2022 9:09:01 PM #5297

Closed
performanceautofiler bot opened this issue May 17, 2022 · 4 comments
Closed

[Perf] Changes at 5/11/2022 9:09:01 PM #5297

performanceautofiler bot opened this issue May 17, 2022 · 4 comments
Assignees
@EgorBo
Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-regression runtime-coreclr

Comments

@performanceautofiler
Copy link

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 130ef3cae8b88593cbd1627d31c2c4bbf6336c40
Compare 7b5f40f08ca09b510151c0c3bcdeec12c61d6427
Diff Diff

Regressions in Benchstone.BenchI.CSieve

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 4.38 ms 4.81 ms 1.10 0.10 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.CSieve*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.CSieve.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.811848611111112 > 4.598636390625.
IsChangePoint: Marked as a change because one of 3/29/2022 11:07:04 AM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -12.120911918514391 (T) = (0 -5468542.42355921) / Math.Sqrt((436082813.0815188 / (37)) + (258937262934.078 / (31))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (31) - 2, .025) and -0.2542569547245288 = (4359985.72936776 - 5468542.42355921) / 4359985.72936776 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline 130ef3cae8b88593cbd1627d31c2c4bbf6336c40
Compare 7b5f40f08ca09b510151c0c3bcdeec12c61d6427
Diff Diff

Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Indexer - Duration of single invocation 9.37 ns 10.83 ns 1.16 0.01 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Primitives.StringSegmentBenchmark.Indexer

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.83002669789214 > 9.808561065998537.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -262.59898810782835 (T) = (0 -10.820976713476417) / Math.Sqrt((0.0008384517167480872 / (37)) + (0.00029093148505472937 / (31))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (31) - 2, .025) and -0.15925400758359287 = (9.334431145105293 - 10.820976713476417) / 9.334431145105293 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 130ef3cae8b88593cbd1627d31c2c4bbf6336c40
Compare 7b5f40f08ca09b510151c0c3bcdeec12c61d6427
Diff Diff

Regressions in System.IO.Tests.StreamReaderReadLineTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadLine - Duration of single invocation 37.65 μs 44.91 μs 1.19 0.25 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.IO.Tests.StreamReaderReadLineTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.StreamReaderReadLineTests.ReadLine(LineLengthRange: [ 9, 32])

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 44.908186904761905 > 40.54548443751441.
IsChangePoint: Marked as a change because one of 3/18/2022 7:38:50 AM, 3/31/2022 1:45:25 PM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -15.680909960751533 (T) = (0 -41937.826490785315) / Math.Sqrt((2019498.2937095359 / (35)) + (762750.8071883292 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.12082877943041037 = (37416.80019324409 - 41937.826490785315) / 37416.80019324409 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 130ef3cae8b88593cbd1627d31c2c4bbf6336c40
Compare 7b5f40f08ca09b510151c0c3bcdeec12c61d6427
Diff Diff

Regressions in System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FixedTimeEquals_256Bit_SameReference - Duration of single invocation 127.90 ns 135.30 ns 1.06 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals*'

Payloads

Baseline
Compare

Histogram

System.Security.Cryptography.Primitives.Tests.Performance.Perf_FixedTimeEquals.FixedTimeEquals_256Bit_SameReference

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 135.3016725349523 > 134.29865954025124.
IsChangePoint: Marked as a change because one of 3/7/2022 10:45:01 PM, 3/11/2022 8:42:43 PM, 3/23/2022 6:35:24 PM, 3/31/2022 1:45:25 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -264.6247235799773 (T) = (0 -135.32078887254482) / Math.Sqrt((0.003718807480868864 / (36)) + (0.021244299417760293 / (31))) is less than -1.997137908391408 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (31) - 2, .025) and -0.05810635847839502 = (127.88959048232378 - 135.32078887254482) / 127.88959048232378 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 130ef3cae8b88593cbd1627d31c2c4bbf6336c40
Compare 7b5f40f08ca09b510151c0c3bcdeec12c61d6427
Diff Diff

Regressions in System.Collections.Sort<IntStruct>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Array - Duration of single invocation 3.89 μs 6.84 μs 1.76 0.24 False
List - Duration of single invocation 4.29 μs 7.67 μs 1.79 0.35 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Sort&lt;IntStruct&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<IntStruct>.Array(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.843093333333333 > 4.075671384615385.
IsChangePoint: Marked as a change because one of 3/15/2022 12:53:39 AM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -90.7837664468618 (T) = (0 -6952.398872503841) / Math.Sqrt((23920.123256378527 / (37)) + (13689.992534227105 / (31))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (31) - 2, .025) and -0.7566457767761491 = (3957.7693832293835 - 6952.398872503841) / 3957.7693832293835 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.Sort&lt;IntStruct&gt;.List(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.6715773333333335 > 4.348428700000001.
IsChangePoint: Marked as a change because one of 3/15/2022 12:53:39 AM, 5/11/2022 9:09:01 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -49.156351307861776 (T) = (0 -7174.9655697191665) / Math.Sqrt((54196.0072207783 / (37)) + (68995.48916725106 / (30))) is less than -1.997137908391408 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (30) - 2, .025) and -0.7252036619347615 = (4158.909309103059 - 7174.9655697191665) / 4158.909309103059 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 130ef3cae8b88593cbd1627d31c2c4bbf6336c40
Compare 7b5f40f08ca09b510151c0c3bcdeec12c61d6427
Diff Diff

Regressions in System.Collections.IndexerSetReverse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IList - Duration of single invocation 1.30 μs 1.45 μs 1.12 0.02 False
Span - Duration of single invocation 246.18 ns 325.39 ns 1.32 0.00 True

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSetReverse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSetReverse<Int32>.IList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4500228831047794 > 1.3588428774779011.
IsChangePoint: Marked as a change because one of 3/7/2022 10:45:01 PM, 3/11/2022 8:42:43 PM, 3/30/2022 6:37:35 PM, 4/8/2022 7:02:49 PM, 4/16/2022 5:36:25 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -230.02455579703468 (T) = (0 -1452.2334374466561) / Math.Sqrt((6.141277802731051 / (37)) + (9.585647973900992 / (31))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (31) - 2, .025) and -0.12257092333280598 = (1293.6674264955248 - 1452.2334374466561) / 1293.6674264955248 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IndexerSetReverse&lt;Int32&gt;.Span(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 325.3875074318889 > 258.4889441544423.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -4091.3621398200744 (T) = (0 -325.39099433106077) / Math.Sqrt((0.012788322070902823 / (37)) + (0.0009153875514273372 / (31))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (31) - 2, .025) and -0.32194671960948 = (246.14531698159925 - 325.39099433106077) / 246.14531698159925 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 130ef3cae8b88593cbd1627d31c2c4bbf6336c40
Compare 7b5f40f08ca09b510151c0c3bcdeec12c61d6427
Diff Diff

Regressions in System.Collections.IterateForEach<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 3.16 μs 3.69 μs 1.17 0.43 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEach&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEach<String>.Dictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.6948585129055176 > 3.2364887166345686.
IsChangePoint: Marked as a change because one of 3/23/2022 6:35:24 PM, 4/8/2022 7:02:49 PM, 4/20/2022 2:49:46 PM, 4/25/2022 2:48:42 AM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -9.005013604272548 (T) = (0 -3555.3317548355753) / Math.Sqrt((32472.370527943844 / (37)) + (19253.962449036695 / (30))) is less than -1.997137908391408 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (30) - 2, .025) and -0.10954416203056865 = (3204.31748145021 - 3555.3317548355753) / 3204.31748145021 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 130ef3cae8b88593cbd1627d31c2c4bbf6336c40
Compare 7b5f40f08ca09b510151c0c3bcdeec12c61d6427
Diff Diff

Regressions in PerfLabTests.LowLevelPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GenericGenericMethod - Duration of single invocation 155.92 μs 187.11 μs 1.20 0.02 False
InstanceDelegate - Duration of single invocation 218.28 μs 249.59 μs 1.14 0.02 False
GenericClassGenericStaticMethod - Duration of single invocation 155.92 μs 187.10 μs 1.20 0.02 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.LowLevelPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.LowLevelPerf.GenericGenericMethod

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 187.1050967261905 > 163.7166584158416.
IsChangePoint: Marked as a change because one of 3/7/2022 10:45:01 PM, 3/11/2022 8:42:43 PM, 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -30142.597166363394 (T) = (0 -187104.4856414254) / Math.Sqrt((24.75614853594859 / (37)) + (12.039486632144108 / (30))) is less than -1.997137908391408 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (30) - 2, .025) and -0.20001162634380704 = (155918.8940622975 - 187104.4856414254) / 155918.8940622975 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.InstanceDelegate

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 249.59332837301585 > 229.2014133029514.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/21/2022 1:28:14 PM, 3/23/2022 6:35:24 PM, 3/31/2022 1:45:25 PM, 4/16/2022 5:36:25 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -7549.890938401556 (T) = (0 -249473.8248279392) / Math.Sqrt((7.787246901068193 / (37)) + (522.4720995951583 / (31))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (31) - 2, .025) and -0.14287641173724708 = (218285.9163649398 - 249473.8248279392) / 218285.9163649398 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.LowLevelPerf.GenericClassGenericStaticMethod

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 187.10191964285715 > 163.7166757425743.
IsChangePoint: Marked as a change because one of 3/7/2022 10:45:01 PM, 3/11/2022 8:42:43 PM, 3/30/2022 6:37:35 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -13295.9356785059 (T) = (0 -187104.5517980697) / Math.Sqrt((169.91577269416803 / (37)) + (27.238070168727752 / (30))) is less than -1.997137908391408 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (30) - 2, .025) and -0.19998738406815897 = (155922.0990838702 - 187104.5517980697) / 155922.0990838702 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 130ef3cae8b88593cbd1627d31c2c4bbf6336c40
Compare 7b5f40f08ca09b510151c0c3bcdeec12c61d6427
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Insert - Duration of single invocation 14.40 ns 16.48 ns 1.14 0.03 False
Join_List - Duration of single invocation 59.35 ns 65.27 ns 1.10 0.05 False
Insert - Duration of single invocation 14.04 ns 15.25 ns 1.09 0.05 False
Join_Array - Duration of single invocation 55.89 ns 62.14 ns 1.11 0.07 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Insert(s1: "dzsdzsDDZSDZSDZSddsz", i: 7, s2: "Test")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.479578887290344 > 14.987172665484787.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/29/2022 11:07:04 AM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -17.28585082085189 (T) = (0 -16.34019962954351) / Math.Sqrt((0.24876834725899574 / (37)) + (0.06286218514853957 / (31))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (31) - 2, .025) and -0.10983131924991525 = (14.723137963512428 - 16.34019962954351) / 14.723137963512428 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Join_List

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 65.26895651027583 > 62.111929997623925.
IsChangePoint: Marked as a change because one of 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -48.251279564329955 (T) = (0 -65.64776182133028) / Math.Sqrt((0.4503549707178522 / (37)) + (0.11570508594946562 / (31))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (31) - 2, .025) and -0.10216214856778917 = (59.56270763483997 - 65.64776182133028) / 59.56270763483997 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Insert(s1: "Test", i: 2, s2: " Test")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.246373055788519 > 14.790327072064867.
IsChangePoint: Marked as a change because one of 3/7/2022 10:45:01 PM, 3/11/2022 8:42:43 PM, 3/17/2022 6:54:53 PM, 3/29/2022 11:07:04 AM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -19.206521043802802 (T) = (0 -15.53053818561355) / Math.Sqrt((0.07549404353389695 / (37)) + (0.04971859782849701 / (30))) is less than -1.997137908391408 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (30) - 2, .025) and -0.08131654086677984 = (14.362619638800975 - 15.53053818561355) / 14.362619638800975 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Join_Array

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 62.14070845837596 > 58.397177866298385.
IsChangePoint: Marked as a change because one of 3/15/2022 11:34:41 AM, 3/17/2022 6:54:53 PM, 3/29/2022 11:07:04 AM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -40.261391493805014 (T) = (0 -62.27092108638368) / Math.Sqrt((0.5440496620381416 / (37)) + (0.280957492477187 / (31))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (31) - 2, .025) and -0.11071176673824665 = (56.06397892880036 - 62.27092108638368) / 56.06397892880036 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 130ef3cae8b88593cbd1627d31c2c4bbf6336c40
Compare 7b5f40f08ca09b510151c0c3bcdeec12c61d6427
Diff Diff

Regressions in PerfLabTests.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CheckObjIsInterfaceNo - Duration of single invocation 249.48 μs 280.66 μs 1.12 0.02 False
CheckIsInstAnyIsInterfaceYes - Duration of single invocation 218.29 μs 280.65 μs 1.29 0.02 False
CheckObjIsInterfaceYes - Duration of single invocation 218.31 μs 280.65 μs 1.29 0.02 False
CheckIsInstAnyIsInterfaceNo - Duration of single invocation 249.47 μs 280.65 μs 1.12 0.02 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.CastingPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.CastingPerf.CheckObjIsInterfaceNo

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 280.65680803571433 > 261.9444362599207.
IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 3/30/2022 6:37:35 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -6501.099310653576 (T) = (0 -280663.54368845443) / Math.Sqrt((53.588663988140596 / (37)) + (647.1701477066086 / (30))) is less than -1.997137908391408 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (30) - 2, .025) and -0.12503315705736004 = (249471.35284666525 - 280663.54368845443) / 249471.35284666525 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf.CheckIsInstAnyIsInterfaceYes

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 280.64848533163274 > 229.20956380208338.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -1532.6558281998896 (T) = (0 -280655.547679447) / Math.Sqrt((59904.839731342145 / (37)) + (1062.0612368990674 / (31))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (31) - 2, .025) and -0.285427413836501 = (218336.3639661296 - 280655.547679447) / 218336.3639661296 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf.CheckObjIsInterfaceYes

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 280.65301339285713 > 229.21572222222224.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -11858.87754334781 (T) = (0 -280652.90440010146) / Math.Sqrt((907.9276400088395 / (36)) + (72.7228142540018 / (30))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (30) - 2, .025) and -0.28562075651274516 = (218301.47263752518 - 280652.90440010146) / 218301.47263752518 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### PerfLabTests.CastingPerf.CheckIsInstAnyIsInterfaceNo

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 280.64972098214287 > 261.9426176167583.
IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 3/30/2022 6:37:35 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -71.25258734716357 (T) = (0 -281099.74869268556) / Math.Sqrt((648.3892558706673 / (37)) + (5909603.761722268 / (30))) is less than -1.997137908391408 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (30) - 2, .025) and -0.1267689598634236 = (249474.1679134983 - 281099.74869268556) / 249474.1679134983 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 130ef3cae8b88593cbd1627d31c2c4bbf6336c40
Compare 7b5f40f08ca09b510151c0c3bcdeec12c61d6427
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfValue - Duration of single invocation 9.57 ns 10.93 ns 1.14 0.09 False
IndexOfAnyFourValues - Duration of single invocation 812.75 ns 890.62 ns 1.10 0.15 False
SequenceEqual - Duration of single invocation 12.71 ns 13.77 ns 1.08 0.07 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.IndexOfValue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.932505463545642 > 10.050008110450669.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -20.51111070488265 (T) = (0 -11.810527115696898) / Math.Sqrt((0.32389998673552817 / (36)) + (0.2636387720331884 / (31))) is less than -1.997137908391408 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (31) - 2, .025) and -0.2982837695427193 = (9.097030551230564 - 11.810527115696898) / 9.097030551230564 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.IndexOfAnyFourValues(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 890.6237243440066 > 856.3959551477573.
IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 3/17/2022 6:54:53 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -19.428651734795483 (T) = (0 -890.0554242131602) / Math.Sqrt((446.06432701471385 / (36)) + (0.3186800192815919 / (30))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (30) - 2, .025) and -0.08327152510804595 = (821.6364997911172 - 890.0554242131602) / 821.6364997911172 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Byte&gt;.SequenceEqual(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.769162260933285 > 13.347345153623428.
IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 3/17/2022 6:54:53 PM, 4/4/2022 8:14:13 PM, 4/14/2022 8:42:05 AM, 5/11/2022 9:09:01 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -40.93016345141407 (T) = (0 -13.788690829718368) / Math.Sqrt((0.01681811890126513 / (37)) + (0.010290759122792493 / (29))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (29) - 2, .025) and -0.09224003753945666 = (12.624231264018517 - 13.788690829718368) / 12.624231264018517 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.18362
Baseline 130ef3cae8b88593cbd1627d31c2c4bbf6336c40
Compare 7b5f40f08ca09b510151c0c3bcdeec12c61d6427
Diff Diff

Regressions in System.Tests.Perf_UInt16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 10.95 ns 12.01 ns 1.10 0.07 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt16.ToString(value: 65535)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.012761077557949 > 11.460847727141498.
IsChangePoint: Marked as a change because one of 3/7/2022 10:45:01 PM, 3/21/2022 1:28:14 PM, 4/25/2022 11:05:49 AM, 5/5/2022 12:13:02 AM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -14.472097867415691 (T) = (0 -12.024529514852242) / Math.Sqrt((0.0572749030469326 / (37)) + (0.07245504836979735 / (30))) is less than -1.997137908391408 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (30) - 2, .025) and -0.0819788459132494 = (11.113460822519945 - 12.024529514852242) / 11.113460822519945 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 130ef3cae8b88593cbd1627d31c2c4bbf6336c40
Compare 7b5f40f08ca09b510151c0c3bcdeec12c61d6427
Diff Diff

Regressions in ByteMark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BenchNumericSortJagged - Duration of single invocation 1.36 secs 1.44 secs 1.06 0.02 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'ByteMark*'

Payloads

Baseline
Compare

Histogram

ByteMark.BenchNumericSortJagged

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4427182333333333 > 1.42327164.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/21/2022 1:28:14 PM, 4/26/2022 12:44:27 PM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -47.79465023787896 (T) = (0 -1431942288.2063491) / Math.Sqrt((35622199489666.41 / (35)) + (37529890395552.29 / (30))) is less than -1.9983405425199077 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (30) - 2, .025) and -0.05293602487009734 = (1359951843.591837 - 1431942288.2063491) / 1359951843.591837 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 130ef3cae8b88593cbd1627d31c2c4bbf6336c40
Compare 7b5f40f08ca09b510151c0c3bcdeec12c61d6427
Diff Diff

Regressions in System.Numerics.Tests.Perf_BigInteger

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Add - Duration of single invocation 57.91 ns 64.69 ns 1.12 0.02 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_BigInteger*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_BigInteger.Add(arguments: 1024,1024 bits)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 64.68541237268018 > 61.647570157709474.
IsChangePoint: Marked as a change because one of 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -18.809201550779903 (T) = (0 -64.98449057234758) / Math.Sqrt((1.181734276475583 / (37)) + (2.3386186570146434 / (29))) is less than -1.997729654317006 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (29) - 2, .025) and -0.1075625374672153 = (58.67342779664139 - 64.98449057234758) / 58.67342779664139 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 130ef3cae8b88593cbd1627d31c2c4bbf6336c40
Compare 7b5f40f08ca09b510151c0c3bcdeec12c61d6427
Diff Diff

Regressions in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParse - Duration of single invocation 13.37 ns 15.35 ns 1.15 0.19 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt64.TryParse(value: "12345")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.348330155198896 > 14.038310258146206.
IsChangePoint: Marked as a change because one of 4/8/2022 2:39:06 AM, 4/16/2022 2:55:52 PM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -11.255742016043072 (T) = (0 -14.680915667834826) / Math.Sqrt((0.18141721013503337 / (37)) + (0.15900878179102598 / (31))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (31) - 2, .025) and -0.08318143720179727 = (13.553514825512805 - 14.680915667834826) / 13.553514825512805 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 130ef3cae8b88593cbd1627d31c2c4bbf6336c40
Compare 7b5f40f08ca09b510151c0c3bcdeec12c61d6427
Diff Diff

Regressions in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastIndexOfValue - Duration of single invocation 17.11 ns 21.11 ns 1.23 0.11 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.LastIndexOfValue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.113407683292163 > 17.918617227519693.
IsChangePoint: Marked as a change because one of 3/28/2022 7:37:41 AM, 4/14/2022 8:42:05 AM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -15.951036792397401 (T) = (0 -19.95869665601817) / Math.Sqrt((0.13956823941362798 / (37)) + (0.7447485552811596 / (31))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (31) - 2, .025) and -0.15372815904405362 = (17.29930616632897 - 19.95869665601817) / 17.29930616632897 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 130ef3cae8b88593cbd1627d31c2c4bbf6336c40
Compare 7b5f40f08ca09b510151c0c3bcdeec12c61d6427
Diff Diff

Regressions in System.Collections.ContainsTrue<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableHashSet - Duration of single invocation 25.47 μs 27.88 μs 1.09 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsTrue&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsTrue<Int32>.ImmutableHashSet(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.87887513638844 > 27.01584722598862.
IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 3/31/2022 1:45:25 PM, 4/8/2022 7:02:49 PM, 4/16/2022 5:36:25 PM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -13.686021192379108 (T) = (0 -27277.24672501443) / Math.Sqrt((95009.4565044954 / (37)) + (253257.50636648855 / (31))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (31) - 2, .025) and -0.05484210368908541 = (25859.080358679344 - 27277.24672501443) / 25859.080358679344 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 130ef3cae8b88593cbd1627d31c2c4bbf6336c40
Compare 7b5f40f08ca09b510151c0c3bcdeec12c61d6427
Diff Diff

Regressions in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 927.37 μs 1.00 ms 1.08 0.04 False
List - Duration of single invocation 918.91 μs 1.07 ms 1.16 0.04 False
ImmutableArray - Duration of single invocation 923.24 μs 1.08 ms 1.17 0.07 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<String>.Queue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0022261607142857 > 972.9594926470587.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/23/2022 6:35:24 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -163.06017405758004 (T) = (0 -1002538.7169289605) / Math.Sqrt((7048346.011554583 / (37)) + (1323909.336877238 / (31))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (31) - 2, .025) and -0.0852391022577932 = (923795.2400012328 - 1002538.7169289605) / 923795.2400012328 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;String&gt;.List(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.0704503869047621 > 963.6869696691176.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/23/2022 6:35:24 PM, 4/16/2022 2:55:52 PM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -19.668488515067406 (T) = (0 -1072402.3958892492) / Math.Sqrt((1475590601.7536747 / (37)) + (1432341.996613056 / (31))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (31) - 2, .025) and -0.13108123358367693 = (948121.4647081431 - 1072402.3958892492) / 948121.4647081431 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.ContainsFalse&lt;String&gt;.ImmutableArray(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.080301875 > 969.1796590686275.
IsChangePoint: Marked as a change because one of 3/29/2022 11:07:04 AM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -19.676051178327327 (T) = (0 -1080598.5763978567) / Math.Sqrt((1496482698.443572 / (37)) + (4689841.3831369 / (31))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (31) - 2, .025) and -0.13124266302128604 = (955231.4562746681 - 1080598.5763978567) / 955231.4562746681 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 130ef3cae8b88593cbd1627d31c2c4bbf6336c40
Compare 7b5f40f08ca09b510151c0c3bcdeec12c61d6427
Diff Diff

Regressions in System.Collections.IndexerSet<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IList - Duration of single invocation 2.24 μs 2.41 μs 1.07 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IndexerSet&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IndexerSet<Int32>.IList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.4050064250835264 > 2.3570795110887097.
IsChangePoint: Marked as a change because one of 3/7/2022 10:45:01 PM, 3/11/2022 8:42:43 PM, 3/21/2022 1:28:14 PM, 3/23/2022 6:35:24 PM, 3/30/2022 6:37:35 PM, 4/8/2022 7:02:49 PM, 4/16/2022 5:36:25 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -1805.9057967981287 (T) = (0 -2405.4222975512666) / Math.Sqrt((0.1326304793289467 / (37)) + (0.1291481490715354 / (30))) is less than -1.997137908391408 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (30) - 2, .025) and -0.07144990092005792 = (2245.016118332478 - 2405.4222975512666) / 2245.016118332478 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline 130ef3cae8b88593cbd1627d31c2c4bbf6336c40
Compare 7b5f40f08ca09b510151c0c3bcdeec12c61d6427
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetUInt64 - Duration of single invocation 578.04 ns 629.60 ns 1.09 0.04 False
GetBoolean - Duration of single invocation 111.73 ns 134.36 ns 1.20 0.05 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Get*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Get.GetUInt64

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 629.5952663742756 > 606.7595446703464.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -51.533460658466694 (T) = (0 -626.8695180796321) / Math.Sqrt((0.37090164904703093 / (36)) + (27.674478380270422 / (31))) is less than -1.997137908391408 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (36) + (31) - 2, .025) and -0.08473998792598053 = (577.8984134974177 - 626.8695180796321) / 577.8984134974177 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Get.GetBoolean

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 134.36298044984503 > 118.22233058992074.
IsChangePoint: Marked as a change because one of 3/14/2022 9:50:33 PM, 3/23/2022 6:35:24 PM, 3/29/2022 11:07:04 AM, 5/11/2022 7:02:26 PM, 5/17/2022 3:30:02 AM falls between 5/8/2022 12:35:08 PM and 5/17/2022 3:30:02 AM.
IsRegressionStdDev: Marked as regression because -28.05036585743814 (T) = (0 -134.2511011269298) / Math.Sqrt((1.2679719831751004 / (37)) + (16.418439355264013 / (31))) is less than -1.9965644189515832 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (37) + (31) - 2, .025) and -0.18609763163032084 = (113.18722636887682 - 134.2511011269298) / 113.18722636887682 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@DrewScoggins DrewScoggins assigned EgorBo and unassigned dakersnar May 17, 2022
@dakersnar
Copy link

Regressions detected in preview 5 perf report for System.Collections.Sort<IntStruct>.List(Size: 512). New historical data seems to show this possibly fixed, but we should monitor this test and see how it progresses, as it looks like it regressed again. Might be bimodal.

image

System.Collections.Sort.List(Size: 512)

Result Ratio Operating System Bit Processor Name Modality
Same 0.99 debian 11 Arm64 Unknown processor several?
Same 0.99 ubuntu 18.04 Arm64 Unknown processor
Same 0.98 ubuntu 20.04 Arm64 Unknown processor
Same 1.09 Windows 11 Arm64 Microsoft SQ1 3.0 GHz
Slower 0.83 macOS Monterey 12.3 Arm64 Apple M1 Max
Slower 0.56 Windows 10 X64 Intel Core i7-6700 CPU 3.40GHz (Skylake)
Slower 0.55 Windows 10 X64 Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R) bimodal
Slower 0.55 Windows 10 X64 Intel Core i9-10900K CPU 3.70GHz
Same 1.04 Windows 11 X64 AMD Ryzen 9 5900X
Slower 0.88 Windows 11 X64 AMD Ryzen 9 5950X
Slower 0.54 Windows 11 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake) bimodal
Same 1.02 Windows 11 X64 11th Gen Intel Core i9-11900H 2.50GHz
Slower 0.54 Windows 11 X64 Intel Core i9-9900T CPU 2.10GHz
Same 1.00 ubuntu 18.04 X64 Intel Xeon CPU E5530 2.40GHz
Slower 0.85 ubuntu 18.04 X64 Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)
Same 0.99 ubuntu 20.04 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Same 1.11 Windows 10 X86 Intel Core i7-6700 CPU 3.40GHz (Skylake)
Same 1.01 macOS Big Sur 11.6.6 X64 Intel Core i5-4278U CPU 2.60GHz (Haswell)

@mrsharm
Copy link
Member

mrsharm commented Jun 8, 2022

@dakersnar: similar pattern detected in System.Collections.Sort<IntStruct>.Array(Size: 512)

image

System.Collections.Sort.Array(Size: 512)

Result Ratio Operating System Bit Processor Name
Same 1.00 debian 11 Arm64 Unknown processor
Same 0.99 ubuntu 18.04 Arm64 Unknown processor
Same 0.96 ubuntu 20.04 Arm64 Unknown processor
Same 1.02 Windows 11 Arm64 Microsoft SQ1 3.0 GHz
Same 1.00 macOS Monterey 12.3 Arm64 Apple M1 Max
Slower 0.55 Windows 10 X64 Intel Core i7-6700 CPU 3.40GHz (Skylake)
Slower 0.60 Windows 10 X64 Intel Core i7-8650U CPU 1.90GHz (Kaby Lake R)
Slower 0.49 Windows 10 X64 Intel Core i9-10900K CPU 3.70GHz
Same 1.09 Windows 11 X64 AMD Ryzen 9 5900X
Faster 1.13 Windows 11 X64 AMD Ryzen 9 5950X
Slower 0.53 Windows 11 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Same 1.05 Windows 11 X64 11th Gen Intel Core i9-11900H 2.50GHz
Slower 0.59 Windows 11 X64 Intel Core i9-9900T CPU 2.10GHz
Same 1.00 ubuntu 18.04 X64 Intel Xeon CPU E5530 2.40GHz
Slower 0.87 ubuntu 18.04 X64 Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)
Same 0.90 ubuntu 20.04 X64 Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Same 0.96 Windows 10 X86 Intel Core i7-6700 CPU 3.40GHz (Skylake)
Same 0.98 macOS Big Sur 11.6.6 X64 Intel Core i5-4278U CPU 2.60GHz (Haswell)

@dakersnar dakersnar mentioned this issue Jun 10, 2022
Closed
14 tasks
@AndyAyersMS
Copy link
Member

Likely from dotnet/runtime#68869, since fixed by dotnet/runtime#70144

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@EgorBo EgorBo

Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-regression runtime-coreclr
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@EgorBo @AndyAyersMS @dakersnar @mrsharm