Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Changes at 5/21/2022 4:54:11 AM #5489

Closed
performanceautofiler bot opened this issue May 24, 2022 · 3 comments
Closed

[Perf] Changes at 5/21/2022 4:54:11 AM #5489

performanceautofiler bot opened this issue May 24, 2022 · 3 comments
Assignees
@kunalspathak
Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-regression runtime-coreclr untriaged

Comments

@performanceautofiler
Copy link

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline f1efc5cf730ed1d7a952d8d6372087f56027c253
Compare f8323092eb414f9fc09a262132c65e09c5427c10
Diff Diff

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WhereAny_LastElementMatches - Duration of single invocation 698.34 ns 828.36 ns 1.19 0.07 False
Repeat - Duration of single invocation 381.27 ns 409.63 ns 1.07 0.04 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

Histogram

System.Linq.Tests.Perf_Enumerable.WhereAny_LastElementMatches(input: IEnumerable)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 828.3602301694407 > 676.7583354052946.
IsChangePoint: Marked as a change because one of 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -17.678882000500465 (T) = (0 -872.5908832651872) / Math.Sqrt((624.7910830121874 / (44)) + (1418.8006383682518 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.2784105825223168 = (682.559183406912 - 872.5908832651872) / 682.559183406912 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Linq.Tests.Perf_Enumerable.Repeat

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 409.62837429315186 > 402.0610908545928.
IsChangePoint: Marked as a change because one of 3/21/2022 1:28:14 PM, 3/23/2022 6:35:24 PM, 3/30/2022 7:10:16 AM, 4/25/2022 11:05:49 AM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -183.63903315488673 (T) = (0 -410.23261926210677) / Math.Sqrt((0.39457663399110793 / (44)) + (0.1816244223635638 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.07101617942139771 = (383.0312063854437 - 410.23261926210677) / 383.0312063854437 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline f1efc5cf730ed1d7a952d8d6372087f56027c253
Compare f8323092eb414f9fc09a262132c65e09c5427c10
Diff Diff

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IsPrefix_DifferentFirstChar - Duration of single invocation 9.97 ns 14.12 ns 1.42 0.05 False
IsPrefix_FirstHalf - Duration of single invocation 12.22 ns 15.89 ns 1.30 0.07 False
IsPrefix_DifferentFirstChar - Duration of single invocation 8.51 ns 12.83 ns 1.51 0.01 True
IsPrefix_FirstHalf - Duration of single invocation 47.36 ns 52.64 ns 1.11 0.02 False
IsPrefix_DifferentFirstChar - Duration of single invocation 17.38 ns 20.69 ns 1.19 0.08 True
IsPrefix_DifferentFirstChar - Duration of single invocation 16.88 ns 21.38 ns 1.27 0.07 True
IsPrefix_DifferentFirstChar - Duration of single invocation 16.90 ns 21.35 ns 1.26 0.10 True
IsPrefix_DifferentFirstChar - Duration of single invocation 17.70 ns 20.87 ns 1.18 0.46 False
IsPrefix_DifferentFirstChar - Duration of single invocation 16.79 ns 21.30 ns 1.27 0.43 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, OrdinalIgnoreCase, False))

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.118083765411939 > 10.604916342151867.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -19.83721457767658 (T) = (0 -13.273285711024544) / Math.Sqrt((0.015159508241602677 / (44)) + (0.36214128595197426 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.3191982296432242 = (10.061630930640568 - 13.273285711024544) / 10.061630930640568 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, Ordinal, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.892353607924713 > 12.968065830948834.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -15.46186619227917 (T) = (0 -14.906340633270004) / Math.Sqrt((0.05111354275575038 / (44)) + (0.4126332820597534 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.22182606812211914 = (12.200051236572687 - 14.906340633270004) / 12.200051236572687 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, Ordinal, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.827113848274227 > 8.88491976015681.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -25.717420653767995 (T) = (0 -11.985371984360556) / Math.Sqrt((0.004324545785819311 / (44)) + (0.25962480811037614 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.41438015234410364 = (8.473939601384227 - 11.985371984360556) / 8.473939601384227 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, OrdinalIgnoreCase, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 52.637586520817294 > 49.71128825302681.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -18.009738268147043 (T) = (0 -51.756330382487945) / Math.Sqrt((0.023962747108719894 / (44)) + (0.818815724711805 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.09235244980682031 = (47.380614554982614 - 51.756330382487945) / 47.380614554982614 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, IgnoreNonSpace, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.685914252457895 > 17.74894767409028.
IsChangePoint: Marked as a change because one of 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -27.03098749050561 (T) = (0 -21.097201022664528) / Math.Sqrt((0.4112757246740319 / (44)) + (0.14592654429782684 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.2197424962080979 = (17.296438459962598 - 21.097201022664528) / 17.296438459962598 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, None, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.382018499372283 > 18.050618642394355.
IsChangePoint: Marked as a change because one of 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -37.34656818477506 (T) = (0 -21.220754691821472) / Math.Sqrt((0.25986358896821005 / (44)) + (0.07434720818432464 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.22908754639166906 = (17.265454160788583 - 21.220754691821472) / 17.265454160788583 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, None, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.34782593715376 > 18.01383551492421.
IsChangePoint: Marked as a change because one of 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -31.68577584083948 (T) = (0 -20.898832551248976) / Math.Sqrt((0.15197846383015673 / (44)) + (0.14492533365095606 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.2167599518967349 = (17.175805727884967 - 20.898832551248976) / 17.175805727884967 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, IgnoreCase, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.865946616531737 > 17.587126612172415.
IsChangePoint: Marked as a change because one of 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -25.288789385134343 (T) = (0 -21.36218076495557) / Math.Sqrt((0.35027650943995203 / (44)) + (0.24317244470576901 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.23214830659707486 = (17.33734539144339 - 21.36218076495557) / 17.33734539144339 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, IgnoreCase, False))

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.29746670465276 > 18.10835626116627.
IsChangePoint: Marked as a change because one of 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -29.877442426540508 (T) = (0 -21.36414598866142) / Math.Sqrt((0.2841793914382375 / (44)) + (0.1917595309882193 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.24772941792064848 = (17.122419077258712 - 21.36414598866142) / 17.122419077258712 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline f1efc5cf730ed1d7a952d8d6372087f56027c253
Compare f8323092eb414f9fc09a262132c65e09c5427c10
Diff Diff

Regressions in System.MathBenchmarks.Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ScaleB - Duration of single invocation 11.14 μs 13.89 μs 1.25 0.03 False
Atan - Duration of single invocation 27.70 μs 30.45 μs 1.10 0.02 False
Exp - Duration of single invocation 19.50 μs 20.90 μs 1.07 0.01 True
Max - Duration of single invocation 5.58 μs 6.97 μs 1.25 0.03 False
Tan - Duration of single invocation 24.39 μs 26.63 μs 1.09 0.02 False

graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Single*'

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Single.ScaleB

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.893231851851853 > 11.699845057547167.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/16/2022 5:36:25 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -389.66085152950643 (T) = (0 -13907.207424186487) / Math.Sqrt((1618.3158795020836 / (44)) + (179.9010703691562 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.24593073289500786 = (11162.103202857923 - 13907.207424186487) / 11162.103202857923 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.MathBenchmarks.Single.Atan

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 30.446411448140903 > 28.963608261460617.
IsChangePoint: Marked as a change because one of 4/16/2022 10:12:41 PM, 4/20/2022 8:23:28 AM, 5/23/2022 3:43:39 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -14.456092966158362 (T) = (0 -30596.771505221623) / Math.Sqrt((267313.15946456837 / (56)) + (45216.29353063436 / (2))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (56) + (2) - 2, .025) and -0.08481239928246223 = (28204.666102138523 - 30596.771505221623) / 28204.666102138523 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.MathBenchmarks.Single.Exp

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.89611463903743 > 20.477962468789013.
IsChangePoint: Marked as a change because one of 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -21.3293737060686 (T) = (0 -20893.77600493868) / Math.Sqrt((169950.01999030833 / (44)) + (8.19433791689092 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.06774810522741607 = (19568.075937244193 - 20893.77600493868) / 19568.075937244193 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.MathBenchmarks.Single.Max

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.970406626506023 > 5.857690122767858.
IsChangePoint: Marked as a change because one of 3/15/2022 12:53:39 AM, 3/17/2022 6:54:53 PM, 3/23/2022 6:35:24 PM, 3/29/2022 11:07:04 AM, 4/16/2022 5:36:25 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -1901.915814213703 (T) = (0 -6973.400749278152) / Math.Sqrt((2.4683337050073875 / (44)) + (6.728737468266279 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.24970634720146487 = (5580.031472908869 - 6973.400749278152) / 5580.031472908869 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.MathBenchmarks.Single.Tan

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.62869235215381 > 25.592403086059193.
IsChangePoint: Marked as a change because one of 5/3/2022 11:42:39 AM, 5/4/2022 7:03:44 PM, 5/21/2022 3:14:39 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -21.551572667640233 (T) = (0 -26624.99540422666) / Math.Sqrt((305263.4159208149 / (47)) + (41.97135167632698 / (11))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (11) - 2, .025) and -0.06980904061216697 = (24887.614885915795 - 26624.99540422666) / 24887.614885915795 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline f1efc5cf730ed1d7a952d8d6372087f56027c253
Compare f8323092eb414f9fc09a262132c65e09c5427c10
Diff Diff

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 19.37 μs 22.75 μs 1.17 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<Int32, Int32>.SortedList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.7487376173097 > 21.208902625773195.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 4/8/2022 7:02:49 PM, 5/11/2022 7:02:26 PM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -26.65665435116439 (T) = (0 -22429.673478838366) / Math.Sqrt((357710.547904909 / (44)) + (54267.79655315532 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.14971807607705873 = (19508.84651250368 - 22429.673478838366) / 19508.84651250368 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline f1efc5cf730ed1d7a952d8d6372087f56027c253
Compare f8323092eb414f9fc09a262132c65e09c5427c10
Diff Diff

Regressions in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 19.63 μs 21.64 μs 1.10 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueTrue<Int32, Int32>.SortedList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.644765406162463 > 20.921200215844255.
IsChangePoint: Marked as a change because one of 3/23/2022 6:35:24 PM, 3/30/2022 6:37:35 PM, 4/16/2022 5:36:25 PM, 4/21/2022 3:06:05 PM, 5/11/2022 9:09:01 PM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -25.29279291121464 (T) = (0 -21902.87236567605) / Math.Sqrt((30793.452055310412 / (44)) + (88255.71894701438 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.10698019881708515 = (19786.146481284286 - 21902.87236567605) / 19786.146481284286 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline f1efc5cf730ed1d7a952d8d6372087f56027c253
Compare f8323092eb414f9fc09a262132c65e09c5427c10
Diff Diff

Regressions in System.Text.Perf_Utf8Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetString - Duration of single invocation 183.48 μs 215.54 μs 1.17 0.00 True
GetString - Duration of single invocation 179.50 μs 211.86 μs 1.18 0.00 True
GetString - Duration of single invocation 292.97 μs 328.05 μs 1.12 0.01 True

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Perf_Utf8Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Utf8Encoding.GetString(Input: Cyrillic)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 215.54119292237442 > 192.27627353533094.
IsChangePoint: Marked as a change because one of 4/22/2022 6:17:37 AM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -25.22981729642912 (T) = (0 -211158.10680807763) / Math.Sqrt((1970974.372340342 / (44)) + (17181974.10734708 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.1557496640352364 = (182702.28699079237 - 211158.10680807763) / 182702.28699079237 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetString(Input: Chinese)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 211.85931306306304 > 189.0486922800619.
IsChangePoint: Marked as a change because one of 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -90.9508837937135 (T) = (0 -211545.51655482015) / Math.Sqrt((1770102.2689875884 / (44)) + (1256103.168415373 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.1834127767427406 = (178758.85803521922 - 211545.51655482015) / 178758.85803521922 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

```#### System.Text.Perf_Utf8Encoding.GetString(Input: Greek)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 328.0490036231884 > 306.90140914351844.
IsChangePoint: Marked as a change because one of 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -29.892816436313016 (T) = (0 -328725.92155866267) / Math.Sqrt((1279984.8286575852 / (44)) + (21301070.079615667 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.12769497199114574 = (291502.5159491832 - 328725.92155866267) / 291502.5159491832 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline f1efc5cf730ed1d7a952d8d6372087f56027c253
Compare f8323092eb414f9fc09a262132c65e09c5427c10
Diff Diff

Regressions in System.Collections.IterateFor<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Span - Duration of single invocation 226.96 ns 296.30 ns 1.31 0.04 False
ReadOnlySpan - Duration of single invocation 224.63 ns 292.67 ns 1.30 0.03 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.IterateFor&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.IterateFor<String>.Span(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 296.3033382085033 > 238.67948815254928.
IsChangePoint: Marked as a change because one of 3/17/2022 3:08:05 PM, 3/23/2022 6:35:24 PM, 3/30/2022 6:37:35 PM, 4/8/2022 7:02:49 PM, 4/16/2022 5:36:25 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -742.3864099946215 (T) = (0 -296.54204999023307) / Math.Sqrt((0.16103847103008018 / (44)) + (0.06960843383488224 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.30309195812358763 = (227.56801478327324 - 296.54204999023307) / 227.56801478327324 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Collections.IterateFor&lt;String&gt;.ReadOnlySpan(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 292.6718112479615 > 235.7950328564542.
IsChangePoint: Marked as a change because one of 3/15/2022 12:53:39 AM, 3/23/2022 6:35:24 PM, 3/30/2022 6:37:35 PM, 4/8/2022 7:02:49 PM, 4/16/2022 5:36:25 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -634.9538999167921 (T) = (0 -293.1228407225121) / Math.Sqrt((0.3478598296627827 / (44)) + (0.05394553939474759 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.3070165013260315 = (224.26866104989858 - 293.1228407225121) / 224.26866104989858 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline f1efc5cf730ed1d7a952d8d6372087f56027c253
Compare f8323092eb414f9fc09a262132c65e09c5427c10
Diff Diff

Regressions in System.MathBenchmarks.Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ScaleB - Duration of single invocation 11.15 μs 14.53 μs 1.30 0.12 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.MathBenchmarks.Double*'

Payloads

Baseline
Compare

Histogram

System.MathBenchmarks.Double.ScaleB

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.534804347826087 > 12.902995846758204.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 5/15/2022 3:27:46 PM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -15.933509966135217 (T) = (0 -14545.82123318685) / Math.Sqrt((546367.8305654454 / (44)) + (257771.29181871444 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.23813671751462046 = (11748.15432530381 - 14545.82123318685) / 11748.15432530381 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline f1efc5cf730ed1d7a952d8d6372087f56027c253
Compare f8323092eb414f9fc09a262132c65e09c5427c10
Diff Diff

Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FirstSpanSingleSegment - Duration of single invocation 6.38 ns 8.35 ns 1.31 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.FirstSpanSingleSegment

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.347672837912501 > 6.637468677652632.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -57.427396321581874 (T) = (0 -8.343019046006708) / Math.Sqrt((0.013339539661814358 / (44)) + (0.011425913671258206 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.2991866881633259 = (6.421724546609481 - 8.343019046006708) / 6.421724546609481 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline f1efc5cf730ed1d7a952d8d6372087f56027c253
Compare f8323092eb414f9fc09a262132c65e09c5427c10
Diff Diff

Regressions in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SequenceEqual - Duration of single invocation 17.60 ns 20.94 ns 1.19 0.08 False
StartsWith - Duration of single invocation 10.48 ns 12.60 ns 1.20 0.08 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.SequenceEqual(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.940791973327674 > 18.461702200065883.
IsChangePoint: Marked as a change because one of 3/23/2022 6:35:24 PM, 4/8/2022 7:02:49 PM, 4/14/2022 8:50:18 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -17.08027717778181 (T) = (0 -20.76501248499202) / Math.Sqrt((0.005238027858244637 / (44)) + (0.497397665860059 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.18385501213686994 = (17.540165199377725 - 20.76501248499202) / 17.540165199377725 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Memory.Span&lt;Char&gt;.StartsWith(Size: 512)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.603634127114347 > 11.911442875942424.
IsChangePoint: Marked as a change because one of 3/23/2022 6:35:24 PM, 4/8/2022 7:02:49 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/9/2022 11:56:12 AM, 5/11/2022 11:37:12 AM, 5/21/2022 3:14:39 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -6.164092695777513 (T) = (0 -12.148350186864677) / Math.Sqrt((0.21552357423722432 / (47)) + (0.45372888391188626 / (11))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (47) + (11) - 2, .025) and -0.12186684742038156 = (10.8286916712073 - 12.148350186864677) / 10.8286916712073 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline f1efc5cf730ed1d7a952d8d6372087f56027c253
Compare f8323092eb414f9fc09a262132c65e09c5427c10
Diff Diff

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Replace_Char - Duration of single invocation 113.57 ns 127.06 ns 1.12 0.02 False
Format_MultipleArgs - Duration of single invocation 222.63 ns 318.87 ns 1.43 0.09 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Replace_Char(text: "yfesgj0sg1ijslnjsb3uofdz3tbzf6ysgblu3at20nfab2wei1kxfbvsbpzwhanjczcqa2psra3aacxb67qnwbnfp2tok6v0a58lzfdql1fehvs91yzkt9xam7ahjbhvpd9edll13ab46i74ktwwgkgbi792e5gkuuzevo5qm8qt83edag7zovoe686gmtw730kms2i5xgji4xcp25287q68fvhwszd3mszht2uh7bchlgkj5qnq1x9m4lg7vwn8cq5l756akua6oyx9k71bmxbysnmhvxvlxde4k9maumfgxd8gxhxx4mwpph2ttyox9zilt3ylv1q9s4bopfuoa8qlrzodg2q67sh85wx4slcd6w7ufnendaxai633ove2ktbaxdt2sz6y6mo42473xd274gz833p6hj3mu77c4m4od9e5s8btxleh0efqnu9zj9rwtbk5758lio35b3q426j5fwwq1qyknfedrsmqyfw1m38mkkotdf7n0vr6p3erhy8dkzntr9fwjrslxjgrbegih0n6bpb5bfuy55bu65ce9kejcfifxwpcs05umrsb8kvd64q2iwugbbi7vd35g5ho0rff9rhombgzzaniyq7bbjbqr88jyw4ccgnoyl31of3a5thv0vg08gnrqzxas800hewtw8tnwgw5pav81ntdpdd62689x3iqpc317y82b3e2trbpdzieoxldaz009tz37gqmh4bdp1bv9lnl5s58udb11z0h7i2sdl5nbyhjyfzxwzezmp4qx0i3eyvsd3fg8sryq9jhlvkonnfcvb4snl4mcbimdzg49tzdhqjmfxfcq3p1st6b9x2xyevo17evpqp4yc4f2rm0f26ivr3t2f5m0boc44vituxaovcqy1jrkcs6im2kdu3jvcexx2k76egve63aon5a6nbxss4rcke90npmqp35qluf571ms160y2nhaqef835wah41qru8tauu362v0r8konl8", oldChar: 'b', newChar: '+')

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 127.06302516936256 > 119.4119711743606.
IsChangePoint: Marked as a change because one of 5/11/2022 9:09:01 PM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -30.912128726416515 (T) = (0 -125.1962527215309) / Math.Sqrt((1.2573791928629143 / (44)) + (1.872753957676123 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.11047549060052868 = (112.74112196193238 - 125.1962527215309) / 112.74112196193238 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Tests.Perf_String.Format_MultipleArgs

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 318.8689420656585 > 234.99865659256483.
IsChangePoint: Marked as a change because one of 4/16/2022 5:36:25 PM, 5/18/2022 10:04:15 AM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -25.86954408941895 (T) = (0 -305.3375336142135) / Math.Sqrt((12.797398595862548 / (33)) + (218.96001854527142 / (25))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (33) + (25) - 2, .025) and -0.3445005606046603 = (227.10108315380285 - 305.3375336142135) / 227.10108315380285 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline f1efc5cf730ed1d7a952d8d6372087f56027c253
Compare f8323092eb414f9fc09a262132c65e09c5427c10
Diff Diff

Regressions in Benchstone.BenchI.Ackermann

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Test - Duration of single invocation 3.11 μs 3.37 μs 1.08 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'Benchstone.BenchI.Ackermann*'

Payloads

Baseline
Compare

Histogram

Benchstone.BenchI.Ackermann.Test

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.3697026827501673 > 3.243922066611322.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -16.917513905649585 (T) = (0 -3343.2375360564397) / Math.Sqrt((768.5083443051317 / (44)) + (2329.802496296878 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.0736731633874284 = (3113.831704155255 - 3343.2375360564397) / 3113.831704155255 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline f1efc5cf730ed1d7a952d8d6372087f56027c253
Compare f8323092eb414f9fc09a262132c65e09c5427c10
Diff Diff

Regressions in System.Collections.ContainsFalse<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Span - Duration of single invocation 693.02 μs 819.46 μs 1.18 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<String>.Span(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 819.46315625 > 728.2633385416667.
IsChangePoint: Marked as a change because one of 4/8/2022 7:02:49 PM, 4/16/2022 5:36:25 PM, 4/25/2022 11:05:49 AM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -47.8102288760353 (T) = (0 -794483.5560325092) / Math.Sqrt((20119778.85789871 / (44)) + (53864632.94027813 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.14266910356215837 = (695287.5102300263 - 794483.5560325092) / 695287.5102300263 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline f1efc5cf730ed1d7a952d8d6372087f56027c253
Compare f8323092eb414f9fc09a262132c65e09c5427c10
Diff Diff

Regressions in System.Buffers.Text.Tests.Utf8FormatterTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
FormatterDecimal - Duration of single invocation 37.49 ns 40.51 ns 1.08 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Utf8FormatterTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8FormatterTests.FormatterDecimal(value: 123456.789)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 40.514584325807576 > 39.62970580779879.
IsChangePoint: Marked as a change because one of 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -18.706690591367934 (T) = (0 -40.585396012055405) / Math.Sqrt((0.23611398351926444 / (44)) + (0.17987718955757398 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.0663330286298367 = (38.06071360670953 - 40.585396012055405) / 38.06071360670953 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline f1efc5cf730ed1d7a952d8d6372087f56027c253
Compare f8323092eb414f9fc09a262132c65e09c5427c10
Diff Diff

Regressions in System.Collections.ContainsFalse<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ImmutableArray - Duration of single invocation 17.69 μs 22.90 μs 1.29 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsFalse&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsFalse<Int32>.ImmutableArray(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.90180200729927 > 18.578962107435316.
IsChangePoint: Marked as a change because one of 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -234.17940377331433 (T) = (0 -22836.113464216007) / Math.Sqrt((20098.93710694968 / (44)) + (755.938168175635 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.3016847409376142 = (17543.50554018707 - 22836.113464216007) / 17543.50554018707 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline f1efc5cf730ed1d7a952d8d6372087f56027c253
Compare f8323092eb414f9fc09a262132c65e09c5427c10
Diff Diff

Regressions in SciMark2.kernel

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
benchSparseMult - Duration of single invocation 446.36 ms 549.12 ms 1.23 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'SciMark2.kernel*'

Payloads

Baseline
Compare

Histogram

SciMark2.kernel.benchSparseMult

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 549.12498 > 468.677085.
IsChangePoint: Marked as a change because one of 3/30/2022 6:37:35 PM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -724.6699442826018 (T) = (0 -548907241.9047619) / Math.Sqrt((13692808121.366623 / (44)) + (276251426874.7152 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.22987285435489782 = (446312185.81751597 - 548907241.9047619) / 446312185.81751597 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline f1efc5cf730ed1d7a952d8d6372087f56027c253
Compare f8323092eb414f9fc09a262132c65e09c5427c10
Diff Diff

Regressions in System.Memory.Slice<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadOnlyMemoryStartLengthSpan - Duration of single invocation 8.95 ns 10.27 ns 1.15 0.06 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Memory.Slice&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Memory.Slice<String>.ReadOnlyMemoryStartLengthSpan

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.270331236369513 > 9.131227110101134.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/30/2022 6:37:35 PM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -31.54267034927705 (T) = (0 -10.304438350337502) / Math.Sqrt((0.0641886702161582 / (44)) + (0.006073925373086946 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.15363156657147783 = (8.93217440379311 - 10.304438350337502) / 8.93217440379311 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline f1efc5cf730ed1d7a952d8d6372087f56027c253
Compare f8323092eb414f9fc09a262132c65e09c5427c10
Diff Diff

Regressions in System.Drawing.Tests.Perf_Color

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetBrightness - Duration of single invocation 754.79 ns 951.24 ns 1.26 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Drawing.Tests.Perf_Color*'

Payloads

Baseline
Compare

Histogram

System.Drawing.Tests.Perf_Color.GetBrightness

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 951.238354037267 > 801.2985651933434.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 4/16/2022 5:36:25 PM, 4/20/2022 2:49:46 PM, 4/25/2022 11:05:49 AM, 5/11/2022 7:02:26 PM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -321.95549570219106 (T) = (0 -951.764229070527) / Math.Sqrt((12.286587856506339 / (44)) + (1.1355938844924458 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.2548047962286333 = (758.4958488611878 - 951.764229070527) / 758.4958488611878 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline f1efc5cf730ed1d7a952d8d6372087f56027c253
Compare f8323092eb414f9fc09a262132c65e09c5427c10
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Get

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetString - Duration of single invocation 3.14 μs 3.59 μs 1.14 0.16 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Get*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Get.GetString

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.593108430210252 > 3.374037302438557.
IsChangePoint: Marked as a change because one of 4/25/2022 9:17:08 AM, 4/26/2022 6:18:08 AM, 4/29/2022 5:18:09 PM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -3.9323597287853844 (T) = (0 -3594.680942076174) / Math.Sqrt((139015.159979428 / (44)) + (1861.1410387095716 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.06697319010197197 = (3369.045235084703 - 3594.680942076174) / 3369.045235084703 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline f1efc5cf730ed1d7a952d8d6372087f56027c253
Compare f8323092eb414f9fc09a262132c65e09c5427c10
Diff Diff

Regressions in System.Formats.Cbor.Tests.Perf_CborReader

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SkipValue - Duration of single invocation 253.34 ns 272.02 ns 1.07 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Formats.Cbor.Tests.Perf_CborReader*'

Payloads

Baseline
Compare

Histogram

System.Formats.Cbor.Tests.Perf_CborReader.SkipValue(encoding: (Array, Strict))

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 272.0176914851962 > 266.14701483335875.
IsChangePoint: Marked as a change because one of 3/11/2022 8:42:43 PM, 3/17/2022 11:24:40 PM, 3/30/2022 6:37:35 PM, 4/25/2022 11:05:49 AM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -8.691456872885366 (T) = (0 -271.5135655751595) / Math.Sqrt((31.318661174290536 / (44)) + (37.75966391762683 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.06281544718393862 = (255.46633359024688 - 271.5135655751595) / 255.46633359024688 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline f1efc5cf730ed1d7a952d8d6372087f56027c253
Compare f8323092eb414f9fc09a262132c65e09c5427c10
Diff Diff

Regressions in System.Collections.ContainsKeyFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 18.94 μs 21.40 μs 1.13 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyFalse<Int32, Int32>.SortedList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.398172146765685 > 21.166710092047435.
IsChangePoint: Marked as a change because one of 3/23/2022 6:35:24 PM, 4/22/2022 6:17:37 AM, 5/11/2022 7:02:26 PM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -20.367338198756187 (T) = (0 -21523.25764251885) / Math.Sqrt((414020.19619236473 / (44)) + (19105.697849084005 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.10892340127168808 = (19409.14730254269 - 21523.25764251885) / 19409.14730254269 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Architecture x64
OS Windows 10.0.19042
Baseline f1efc5cf730ed1d7a952d8d6372087f56027c253
Compare f8323092eb414f9fc09a262132c65e09c5427c10
Diff Diff

Regressions in System.Collections.ContainsKeyTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SortedList - Duration of single invocation 18.98 μs 22.21 μs 1.17 0.05 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.ContainsKeyTrue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.ContainsKeyTrue<Int32, Int32>.SortedList(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.21138045702592 > 19.692823344515805.
IsChangePoint: Marked as a change because one of 3/17/2022 6:54:53 PM, 3/23/2022 6:35:24 PM, 4/16/2022 5:36:25 PM, 4/21/2022 3:06:05 PM, 5/11/2022 9:09:01 PM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -18.897477503534844 (T) = (0 -21627.461895641372) / Math.Sqrt((199584.6949840379 / (44)) + (178493.46038977156 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.12978877106256892 = (19142.92516405584 - 21627.461895641372) / 19142.92516405584 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS Windows 10.0.19042
Baseline f1efc5cf730ed1d7a952d8d6372087f56027c253
Compare f8323092eb414f9fc09a262132c65e09c5427c10
Diff Diff

Regressions in System.Numerics.Tests.Perf_VectorOf<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetHashCodeBenchmark - Duration of single invocation 87.93 ns 98.00 ns 1.11 0.16 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Byte&gt;*'

Payloads

Baseline
Compare

Histogram

System.Numerics.Tests.Perf_VectorOf<Byte>.GetHashCodeBenchmark

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 98.00293909086994 > 93.33842580446017.
IsChangePoint: Marked as a change because one of 3/17/2022 11:30:59 AM, 3/30/2022 6:37:35 PM, 4/22/2022 7:11:21 PM, 4/25/2022 11:05:49 AM, 5/20/2022 10:18:01 PM, 5/24/2022 7:58:36 AM falls between 5/15/2022 6:57:22 PM and 5/24/2022 7:58:36 AM.
IsRegressionStdDev: Marked as regression because -11.157715288527955 (T) = (0 -104.50286464185663) / Math.Sqrt((4.840262531677132 / (44)) + (22.715396260151103 / (14))) is less than -2.0032407188469383 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (44) + (14) - 2, .025) and -0.16351604467138708 = (89.81643624121357 - 104.50286464185663) / 89.81643624121357 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@kunalspathak
Copy link
Member

Dup of dotnet/runtime#69727

@kunalspathak kunalspathak mentioned this issue May 24, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kunalspathak kunalspathak

Labels
arch-x64 branch-refs/heads/main kind-micro os-windows perf-regression runtime-coreclr untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kunalspathak