Skip to content

[Perf] Windows/x64: 9 Regressions on 12/25/2022 7:48:20 PM #11132

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8355deb78ce4e0c14689ce5eac9ea9fc480faeb0
Compare bd6709248295deefa1956da3aeb9b0e086fbaca5
Diff Diff

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsPrefix_FirstHalf - Duration of single invocation 6.70 μs 7.21 μs 1.08 0.07 False
IsPrefix_FirstHalf - Duration of single invocation 3.71 μs 4.50 μs 1.21 0.07 False
IsPrefix_FirstHalf - Duration of single invocation 3.73 μs 4.54 μs 1.22 0.08 False
IsPrefix_FirstHalf - Duration of single invocation 3.71 μs 4.52 μs 1.22 0.11 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (pl-PL, None, False))


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.2113108077099595 > 7.169088031841694.
IsChangePoint: Marked as a change because one of 12/23/2022 9:40:21 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -3.820433181418042 (T) = (0 -7354.518873031248) / Math.Sqrt((8613.759996065355 / (47)) + (52690.940800709315 / (3))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (3) - 2, .025) and -0.07434717374211293 = (6845.570084588534 - 7354.518873031248) / 6845.570084588534 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, None, True))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.496179652149322 > 3.8901280101051294.
IsChangePoint: Marked as a change because one of 11/1/2022 6:58:35 AM, 11/1/2022 4:32:17 PM, 12/23/2022 9:40:21 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -22.59696461322638 (T) = (0 -4569.610223041637) / Math.Sqrt((3435.3182118894956 / (47)) + (4054.05945064536 / (3))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (3) - 2, .025) and -0.22946078554309068 = (3716.759637049424 - 4569.610223041637) / 3716.759637049424 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, IgnoreCase, True))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.541652772583729 > 3.902149216877937.
IsChangePoint: Marked as a change because one of 11/1/2022 6:58:35 AM, 11/1/2022 4:32:17 PM, 11/18/2022 1:00:00 AM, 11/18/2022 3:21:53 PM, 12/23/2022 9:40:21 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -74.30572370416776 (T) = (0 -4527.918905270303) / Math.Sqrt((2838.6943547020524 / (47)) + (164.77439704489302 / (3))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (3) - 2, .025) and -0.21393161587794024 = (3729.962088511567 - 4527.918905270303) / 3729.962088511567 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (, IgnoreCase, True))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.5240537595785435 > 3.902585473656288.
IsChangePoint: Marked as a change because one of 11/1/2022 6:58:35 AM, 11/1/2022 4:32:17 PM, 11/17/2022 7:01:39 PM, 11/18/2022 3:21:53 PM, 12/23/2022 9:40:21 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -14.160563972057991 (T) = (0 -4563.96731122616) / Math.Sqrt((7674.012509234451 / (47)) + (9413.873970413792 / (3))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (3) - 2, .025) and -0.21694350532787737 = (3750.352659137208 - 4563.96731122616) / 3750.352659137208 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 8355deb78ce4e0c14689ce5eac9ea9fc480faeb0
Compare bd6709248295deefa1956da3aeb9b0e086fbaca5
Diff Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableArray - Duration of single invocation 18.14 μs 19.07 μs 1.05 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<Int32>.ImmutableArray(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.066574632352946 > 18.87276251657825.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/17/2022 3:19:47 PM, 12/23/2022 9:40:21 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -4.019249990180762 (T) = (0 -19912.8004589704) / Math.Sqrt((35158.048807095605 / (47)) + (604663.5986493295 / (3))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (3) - 2, .025) and -0.09984969009208314 = (18105.019838941113 - 19912.8004589704) / 18105.019838941113 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8355deb78ce4e0c14689ce5eac9ea9fc480faeb0
Compare bd6709248295deefa1956da3aeb9b0e086fbaca5
Diff Diff

Regressions in System.Text.Json.Reader.Tests.Perf_Base64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadBase64EncodedByteArray_NoEscaping - Duration of single invocation 94.81 ns 129.78 ns 1.37 0.15 False
ReadBase64EncodedByteArray_HeavyEscaping - Duration of single invocation 94.44 ns 128.90 ns 1.36 0.14 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Reader.Tests.Perf_Base64*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_NoEscaping(NumberOfBytes: 100)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 129.78486574630764 > 99.44180681786246.
IsChangePoint: Marked as a change because one of 10/11/2022 8:42:28 PM, 10/14/2022 9:06:25 AM, 11/22/2022 2:20:42 AM, 12/23/2022 9:40:21 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -25.55028063016023 (T) = (0 -129.93224427691712) / Math.Sqrt((53.63251079594049 / (47)) + (0.08700826337928207 / (3))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (3) - 2, .025) and -0.27018474884920257 = (102.29397290011296 - 129.93224427691712) / 102.29397290011296 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Reader.Tests.Perf_Base64.ReadBase64EncodedByteArray_HeavyEscaping(NumberOfBytes: 100)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 128.9025521921711 > 99.19288937337053.
IsChangePoint: Marked as a change because one of 11/2/2022 9:35:40 AM, 11/10/2022 9:13:35 AM, 12/6/2022 2:12:43 AM, 12/17/2022 10:44:50 PM, 12/23/2022 9:40:21 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -28.010330174427953 (T) = (0 -128.95267610928155) / Math.Sqrt((42.78397963843383 / (47)) + (0.0050094813851160784 / (3))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (3) - 2, .025) and -0.2617227141621575 = (102.20365747708055 - 128.95267610928155) / 102.20365747708055 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8355deb78ce4e0c14689ce5eac9ea9fc480faeb0
Compare bd6709248295deefa1956da3aeb9b0e086fbaca5
Diff Diff

Regressions in System.Diagnostics.Tracing.Perf_EventListener

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteEvent_IntParams - Duration of single invocation 68.70 ns 73.60 ns 1.07 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Diagnostics.Tracing.Perf_EventListener*'

Payloads

Baseline
Compare

Histogram

System.Diagnostics.Tracing.Perf_EventListener.WriteEvent_IntParams


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 73.5963348983555 > 72.48026579767752.
IsChangePoint: Marked as a change because one of 11/11/2022 5:53:56 PM, 11/18/2022 7:31:41 AM, 12/23/2022 9:40:21 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -59.66874064015645 (T) = (0 -73.6334363289504) / Math.Sqrt((0.1409088059576603 / (47)) + (0.006967182574554933 / (3))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (3) - 2, .025) and -0.062821219343263 = (69.28111237226696 - 73.6334363289504) / 69.28111237226696 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 8355deb78ce4e0c14689ce5eac9ea9fc480faeb0
Compare bd6709248295deefa1956da3aeb9b0e086fbaca5
Diff Diff

Regressions in System.Collections.CreateAddAndRemove<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
List - Duration of single invocation 40.70 μs 45.35 μs 1.11 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.CreateAddAndRemove&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.CreateAddAndRemove<String>.List(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 45.352141861761424 > 42.71735537862728.
IsChangePoint: Marked as a change because one of 10/27/2022 8:00:27 PM, 11/10/2022 12:29:27 AM, 12/3/2022 7:24:45 AM, 12/6/2022 2:12:43 AM, 12/23/2022 9:40:21 PM, 12/26/2022 8:24:36 PM falls between 12/17/2022 10:44:50 PM and 12/26/2022 8:24:36 PM.
IsRegressionStdDev: Marked as regression because -41.5431540063143 (T) = (0 -45477.21757492219) / Math.Sqrt((27284.09967984074 / (47)) + (38163.613939340685 / (3))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (3) - 2, .025) and -0.11776309320080697 = (40685.91801925972 - 45477.21757492219) / 40685.91801925972 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions