Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Changes at 3/31/2022 1:23:38 AM #4393

Closed
performanceautofiler bot opened this issue Apr 5, 2022 · 1 comment
Closed

[Perf] Changes at 3/31/2022 1:23:38 AM #4393

performanceautofiler bot opened this issue Apr 5, 2022 · 1 comment
Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-coreclr

Comments

@performanceautofiler
Copy link

performanceautofiler bot commented Apr 5, 2022 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Substring_IntInt - Duration of single invocation 6.89 ns 7.98 ns 1.16 0.06 False
Trim_CharArr - Duration of single invocation 13.63 ns 15.63 ns 1.15 0.11 False
Trim - Duration of single invocation 10.13 ns 12.57 ns 1.24 0.14 False
GetChars - Duration of single invocation 15.60 ns 16.61 ns 1.06 0.10 False

graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Substring_IntInt(s: "dzsdzsDDZSDZSDZSddsz", i1: 0, i2: 8)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.979856890811089 > 7.195203123817243.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -38.186581978756465 (T) = (0 -7.99738308860944) / Math.Sqrt((0.0234444841934729 / (32)) + (0.0018153406554687929 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.15707418981489626 = (6.911728875301269 - 7.99738308860944) / 6.911728875301269 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim_CharArr(s: "Test ", c: [' ', ' '])

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.630618591392677 > 14.299999392777003.
IsChangePoint: Marked as a change because one of 2/19/2022 11:51:48 PM, 3/7/2022 10:58:04 AM, 3/7/2022 10:45:01 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -23.434866031960432 (T) = (0 -15.828645855124867) / Math.Sqrt((0.03708572567245227 / (32)) + (0.16953719800524428 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.1550756985226269 = (13.703557156790792 - 15.828645855124867) / 13.703557156790792 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Trim(s: "Test ")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.572419893921218 > 10.69869936761602.
IsChangePoint: Marked as a change because one of 2/10/2022 7:02:09 PM, 2/15/2022 5:17:31 AM, 3/7/2022 10:45:01 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -47.16910469584805 (T) = (0 -12.716428606006675) / Math.Sqrt((0.026552719633654787 / (32)) + (0.04593644787803155 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.24115345043824352 = (10.245653832341668 - 12.716428606006675) / 10.245653832341668 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.GetChars(size: 100)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.611589968937135 > 16.466306833530062.
IsChangePoint: Marked as a change because one of 2/4/2022 6:00:25 PM, 2/8/2022 5:20:12 PM, 2/15/2022 5:17:31 AM, 2/19/2022 11:51:48 PM, 2/22/2022 10:11:12 PM, 3/11/2022 8:42:43 PM, 3/17/2022 6:54:53 PM, 3/29/2022 11:07:04 AM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -10.642487396930125 (T) = (0 -16.85860923844707) / Math.Sqrt((0.04418948176116858 / (25)) + (0.30123973014413524 / (31))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (25) + (31) - 2, .025) and -0.07256197689104733 = (15.718074667641883 - 16.85860923844707) / 15.718074667641883 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Regressions in System.Tests.Perf_Int64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ParseSpan - Duration of single invocation 26.85 ns 28.84 ns 1.07 0.04 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_Int64*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Int64.ParseSpan(value: "-9223372036854775808")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.836499971782906 > 28.252110565577855.
IsChangePoint: Marked as a change because one of 3/3/2022 10:59:13 PM, 3/11/2022 8:42:43 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -11.456857520583215 (T) = (0 -28.122979633931664) / Math.Sqrt((0.18776665543562202 / (32)) + (0.2073576694803045 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.05160037543857874 = (26.743029282585354 - 28.122979633931664) / 26.743029282585354 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FormatterDouble - Duration of single invocation 211.85 ns 229.12 ns 1.08 0.09 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDouble(value: -1.7976931348623157E+308)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 229.1235802031307 > 222.24873770506574.
IsChangePoint: Marked as a change because one of 2/1/2022 8:13:55 AM, 2/4/2022 6:00:25 PM, 2/19/2022 11:51:48 PM, 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -18.88422649319888 (T) = (0 -227.90351027659707) / Math.Sqrt((26.464078153077583 / (32)) + (5.9403293269363715 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.09396309988772104 = (208.3283342006581 - 227.90351027659707) / 208.3283342006581 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Regressions in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse - Duration of single invocation 13.65 ns 15.99 ns 1.17 0.12 False
TryParse - Duration of single invocation 10.94 ns 12.16 ns 1.11 0.18 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt32.Parse(value: "12345")

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.986449933207306 > 14.303695476832976.
IsChangePoint: Marked as a change because one of 2/11/2022 3:03:11 AM, 2/19/2022 11:51:48 PM, 3/3/2022 10:59:13 PM, 3/10/2022 7:26:59 AM, 3/11/2022 8:42:43 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -66.51122361089854 (T) = (0 -16.01376558836363) / Math.Sqrt((0.01061074796175624 / (32)) + (0.023877991572006402 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.17823228502937294 = (13.591348490305892 - 16.01376558836363) / 13.591348490305892 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt32.TryParse(value: "0")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.163937832879936 > 11.549082873674744.
IsChangePoint: Marked as a change because one of 2/11/2022 5:21:58 PM, 2/22/2022 2:43:53 PM, 3/4/2022 1:10:06 AM, 3/10/2022 7:26:59 AM, 3/31/2022 1:45:25 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -39.8875084901835 (T) = (0 -12.043908179207323) / Math.Sqrt((0.006965389881653648 / (35)) + (0.010501563360190097 / (21))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (35) + (21) - 2, .025) and -0.09596947178810915 = (10.98927341430168 - 12.043908179207323) / 10.98927341430168 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Regressions in System.MathBenchmarks.Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ScaleB - Duration of single invocation 11.17 μs 12.85 μs 1.15 0.08 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Double*'

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Double.ScaleB

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.853292140385568 > 11.742843493080084.
IsChangePoint: Marked as a change because one of 3/4/2022 3:52:45 PM, 3/7/2022 10:45:01 PM, 3/11/2022 8:42:43 PM, 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -26.81041980930267 (T) = (0 -12770.079823026732) / Math.Sqrt((30127.756883334383 / (32)) + (57244.752157639094 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.1377757806324883 = (11223.722670496518 - 12770.079823026732) / 11223.722670496518 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TrimStart - Duration of single invocation 8.86 ns 10.08 ns 1.14 0.15 False
TrimEnd - Duration of single invocation 9.05 ns 10.24 ns 1.13 0.06 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Primitives.StringSegmentBenchmark.TrimStart

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.077224439618442 > 9.301841512591974.
IsChangePoint: Marked as a change because one of 2/1/2022 8:13:55 AM, 2/19/2022 11:51:48 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -17.201079535862362 (T) = (0 -9.759062513375023) / Math.Sqrt((7.945832199230686E-06 / (32)) + (0.06567444160681053 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.10157168190574195 = (8.859216947635801 - 9.759062513375023) / 8.859216947635801 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### Microsoft.Extensions.Primitives.StringSegmentBenchmark.TrimEnd

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.243934648439415 > 9.472209234089274.
IsChangePoint: Marked as a change because one of 2/25/2022 3:59:26 PM, 3/23/2022 6:35:24 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -20.897648730359776 (T) = (0 -10.250986521093326) / Math.Sqrt((0.0005688070440712707 / (32)) + (0.0844125489774111 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.13792317609935165 = (9.008504911757168 - 10.250986521093326) / 9.008504911757168 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Regressions in PerfLabTests.BlockCopyPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
CallBlockCopy - Duration of single invocation 6.97 ns 9.27 ns 1.33 0.07 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'PerfLabTests.BlockCopyPerf*'

Payloads

Baseline
Compare

Histogram

PerfLabTests.BlockCopyPerf.CallBlockCopy(numElements: 100)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.271200496914744 > 7.311104272800735.
IsChangePoint: Marked as a change because one of 2/10/2022 7:02:09 PM, 2/15/2022 8:18:05 AM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -80.42712189148945 (T) = (0 -9.264917148679087) / Math.Sqrt((0.011192921682573973 / (32)) + (0.010099545163465794 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.31748118354618315 = (7.032295614075701 - 9.264917148679087) / 7.032295614075701 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SequenceEqual - Duration of single invocation 9.24 ns 12.73 ns 1.38 0.26 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Byte>.SequenceEqual(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.7285276973223 > 9.778459176149527.
IsChangePoint: Marked as a change because one of 2/1/2022 8:13:55 AM, 2/19/2022 11:51:48 PM, 2/23/2022 5:12:02 AM, 3/11/2022 8:42:43 PM, 3/17/2022 6:54:53 PM, 3/23/2022 6:35:24 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -23.6820503067425 (T) = (0 -12.698538444086674) / Math.Sqrt((0.5242632090174766 / (32)) + (0.0036394544500370062 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.3154593226461385 = (9.653311376092326 - 12.698538444086674) / 9.653311376092326 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Regressions in System.Tests.Perf_UInt16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 10.07 ns 11.21 ns 1.11 0.14 False
Parse - Duration of single invocation 13.98 ns 16.07 ns 1.15 0.05 False
Parse - Duration of single invocation 9.66 ns 11.28 ns 1.17 0.15 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_UInt16*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_UInt16.ToString(value: 12345)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.21423350784234 > 10.58460267325235.
IsChangePoint: Marked as a change because one of 2/25/2022 7:19:51 AM, 3/9/2022 10:22:25 AM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -16.631670180630056 (T) = (0 -11.309157351938774) / Math.Sqrt((0.11159607942441631 / (32)) + (0.019899810446043272 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.10695559640423731 = (10.216450767017852 - 11.309157351938774) / 10.216450767017852 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt16.Parse(value: "65535")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.071455443696557 > 14.581397366788106.
IsChangePoint: Marked as a change because one of 1/28/2022 6:30:08 PM, 3/3/2022 10:59:13 PM, 3/11/2022 8:42:43 PM, 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -65.32315543430435 (T) = (0 -16.25366518911982) / Math.Sqrt((0.011858464420860537 / (32)) + (0.02175247510689265 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.16769893261076427 = (13.91939714527233 - 16.25366518911982) / 13.91939714527233 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_UInt16.Parse(value: "0")

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.279661892037977 > 10.158791603098441.
IsChangePoint: Marked as a change because one of 1/28/2022 6:30:08 PM, 3/3/2022 10:59:13 PM, 3/11/2022 8:42:43 PM, 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -28.287184442663442 (T) = (0 -11.311681074352641) / Math.Sqrt((0.030434129925414742 / (32)) + (0.05368062782463386 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.1644025214500077 = (9.714579680114765 - 11.311681074352641) / 9.714579680114765 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Regressions in Benchstone.BenchF.Simpsn

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 120.18 ms 130.66 ms 1.09 0.00 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchF.Simpsn*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchF.Simpsn.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 130.660655 > 126.2068185.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -76.768903586732 (T) = (0 -130752172.08333333) / Math.Sqrt((572851206405.3629 / (32)) + (10120952923.188404 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.08633828877131107 = (120360456.25457875 - 130752172.08333333) / 120360456.25457875 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Regressions in System.Collections.IterateForEachNonGeneric<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 3.88 μs 4.32 μs 1.11 0.16 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateForEachNonGeneric&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateForEachNonGeneric<Int32>.Queue(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.317846185288786 > 4.032422873012083.
IsChangePoint: Marked as a change because one of 2/19/2022 11:51:48 PM, 3/12/2022 4:35:06 AM, 3/17/2022 6:54:53 PM, 3/21/2022 1:28:14 PM, 3/23/2022 6:35:24 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -14.404164059699061 (T) = (0 -4247.78770791455) / Math.Sqrt((10856.779712259711 / (32)) + (11875.699879824435 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.1085662168286343 = (3831.785276721261 - 4247.78770791455) / 3831.785276721261 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 13.57 ns 15.54 ns 1.15 0.31 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<UInt64>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.541856714633441 > 14.254422245358022.
IsChangePoint: Marked as a change because one of 3/4/2022 6:17:23 AM, 3/7/2022 10:45:01 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -56.770583323455924 (T) = (0 -15.503191408281559) / Math.Sqrt((0.03534997159659086 / (31)) + (0.003279113407905801 / (24))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (24) - 2, .025) and -0.15055565056657594 = (13.474525461369224 - 15.503191408281559) / 13.474525461369224 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

performanceautofiler bot commented Apr 5, 2022 •
edited
Loading

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Reader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadSingleSpanSequenceEmptyLoop - Duration of single invocation 117.82 ns 136.35 ns 1.16 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Reader*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Reader.ReadSingleSpanSequenceEmptyLoop(IsDataCompact: False, TestCase: HelloWorld)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 136.3519861876974 > 123.80579659523124.
IsChangePoint: Marked as a change because one of 2/3/2022 10:14:29 PM, 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -407.9020898823926 (T) = (0 -136.25670385774595) / Math.Sqrt((0.01810369372823003 / (32)) + (0.03479046639836294 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.15525699043780142 = (117.94492912447947 - 136.25670385774595) / 117.94492912447947 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 13.57 ns 15.52 ns 1.14 0.31 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Int64>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.5245102035645 > 14.25033416902637.
IsChangePoint: Marked as a change because one of 2/10/2022 7:02:09 PM, 2/12/2022 1:03:18 PM, 2/24/2022 3:52:46 AM, 2/26/2022 10:27:55 AM, 3/4/2022 3:52:45 PM, 3/11/2022 8:42:43 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -51.05708339154248 (T) = (0 -15.485723222974352) / Math.Sqrt((0.047708542280315874 / (32)) + (0.003579829901297838 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.15409756675217529 = (13.418036454711348 - 15.485723222974352) / 13.418036454711348 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitArrayRightShift - Duration of single invocation 5.01 ns 6.51 ns 1.30 0.08 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArrayRightShift(Size: 4)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.508646791511391 > 5.265491387696679.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 2/24/2022 10:17:16 AM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -111.78783446525391 (T) = (0 -6.51301136608729) / Math.Sqrt((2.906687380171103E-06 / (32)) + (0.004317214634360627 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.2991399952075183 = (5.013325269111535 - 6.51301136608729) / 5.013325269111535 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Regressions in System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsMatch - Duration of single invocation 77.55 ns 95.66 ns 1.23 0.17 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple*'

Payloads

Baseline
Compare

Histogram

System.Text.RegularExpressions.Tests.Perf_Regex_Industry_BoostDocs_Simple.IsMatch(Id: 2, Options: Compiled)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 95.66495761855434 > 82.05079083600164.
IsChangePoint: Marked as a change because one of 2/28/2022 8:13:22 PM, 3/14/2022 9:50:33 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -10.803827244231087 (T) = (0 -91.43264674244061) / Math.Sqrt((5.8002184394978205 / (26)) + (23.769633363598427 / (24))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (26) + (24) - 2, .025) and -0.14964317648787828 = (79.53132642579092 - 91.43264674244061) / 79.53132642579092 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Regressions in System.Globalization.Tests.StringEquality

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Compare_DifferentFirstChar - Duration of single invocation 13.59 ns 15.19 ns 1.12 0.10 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringEquality*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringEquality.Compare_DifferentFirstChar(Count: 1024, Options: (en-US, Ordinal))

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.192450446203138 > 14.251866592528378.
IsChangePoint: Marked as a change because one of 2/10/2022 10:02:40 PM, 2/26/2022 10:27:55 AM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -10.654312725383354 (T) = (0 -15.456701098744226) / Math.Sqrt((0.11714675936604403 / (32)) + (0.8933534563365838 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.16194621375091797 = (13.302423912418394 - 15.456701098744226) / 13.302423912418394 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Regressions in SciMark2.kernel

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
benchSparseMult - Duration of single invocation 406.41 ms 446.49 ms 1.10 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'SciMark2.kernel*'

Payloads

Baseline
Compare

Histogram

SciMark2.kernel.benchSparseMult

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 446.4947416666667 > 427.1635095.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -71.88078608707274 (T) = (0 -446814501.5018315) / Math.Sqrt((1952296607579.8845 / (32)) + (5775921376714.062 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.09692369250743983 = (407334169.69093406 - 446814501.5018315) / 407334169.69093406 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Regressions in System.Text.Encodings.Web.Tests.Perf_Encoders

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
EncodeUtf16 - Duration of single invocation 15.85 ns 17.99 ns 1.13 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Encodings.Web.Tests.Perf_Encoders*'

Payloads

Baseline
Compare

Histogram

System.Text.Encodings.Web.Tests.Perf_Encoders.EncodeUtf16(arguments: UnsafeRelaxed,no <escaping /> required,16)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.991290508080187 > 17.51685698183491.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -19.7993605163089 (T) = (0 -18.10035498648344) / Math.Sqrt((0.13897715133316282 / (32)) + (0.06731678688559226 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.1019055247074731 = (16.426412773716336 - 18.10035498648344) / 16.426412773716336 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseInt16 - Duration of single invocation 6.63 ns 8.40 ns 1.27 0.03 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt16(value: 32767)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.404305592619918 > 6.961654083641832.
IsChangePoint: Marked as a change because one of 2/1/2022 8:13:55 AM, 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -74.68200190167379 (T) = (0 -8.352464290228442) / Math.Sqrt((0.012379056847083039 / (31)) + (0.0023041304200746428 / (24))) is less than -2.005745995316835 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (31) + (24) - 2, .025) and -0.248436234585656 = (6.69034113143996 - 8.352464290228442) / 6.69034113143996 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline 5a0564b01442f8ea9247e27c4fab85ee0d457265
Compare 43ab6b874e08f55fe88081f75ddde8e4a2fe726e
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
LastWithPredicate_FirstElementMatches - Duration of single invocation 11.86 ns 14.19 ns 1.20 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.LastWithPredicate_FirstElementMatches(input: IList)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.186351106826983 > 12.446980132715717.
IsChangePoint: Marked as a change because one of 2/8/2022 5:20:12 PM, 2/10/2022 7:02:09 PM, 3/30/2022 6:37:35 PM, 4/5/2022 2:02:34 AM falls between 3/26/2022 9:07:40 PM and 4/5/2022 2:02:34 AM.
IsRegressionStdDev: Marked as regression because -39.88411201983 (T) = (0 -14.077526999023688) / Math.Sqrt((0.06432191271691638 / (32)) + (0.017828061877527508 / (24))) is less than -2.0048792881871513 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (32) + (24) - 2, .025) and -0.1746067024546119 = (11.984885638405983 - 14.077526999023688) / 11.984885638405983 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler performanceautofiler bot added CoreClr PGO Applied if there were any profile guided optimization updates in the observed interval. untriaged labels Apr 5, 2022
@EgorBo EgorBo mentioned this issue Apr 5, 2022
Closed
@EgorBo EgorBo removed the untriaged label Apr 5, 2022
@EgorBo EgorBo closed this as completed Apr 5, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-regression PGO Applied if there were any profile guided optimization updates in the observed interval. runtime-coreclr
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@EgorBo