Skip to content

[Perf] Alpine/x64: 11 Regressions on 10/25/2022 12:27:43 PM #9435

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS alpine 3.15
Baseline 5108757b997c59ab8ba1fc5309ab0d4e730e2b77
Compare f4b715f1cfadcbf85486f369724528c051be5733
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Deep

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteDeepUtf8 - Duration of single invocation 22.69 ms 24.28 ms 1.07 0.00 False
WriteDeepUtf16 - Duration of single invocation 22.60 ms 24.13 ms 1.07 0.00 False
WriteDeepUtf16 - Duration of single invocation 3.03 ms 3.57 ms 1.18 0.00 False
WriteDeepUtf8 - Duration of single invocation 3.35 ms 3.70 ms 1.10 0.06 False
WriteDeepUtf16 - Duration of single invocation 22.60 ms 24.25 ms 1.07 0.00 False
WriteDeepUtf8 - Duration of single invocation 3.04 ms 3.55 ms 1.17 0.06 False
WriteDeepUtf16 - Duration of single invocation 3.22 ms 3.66 ms 1.14 0.00 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Deep*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: True, SkipValidation: False)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 24.284502265306124 > 23.80486186166667.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 11/1/2022 10:15:31 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 10:15:31 AM.
IsRegressionStdDev: Marked as regression because -59.030999409149366 (T) = (0 -24402672.47476333) / Math.Sqrt((10410309944.617916 / (19)) + (8541981146.9547415 / (33))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (33) - 2, .025) and -0.07377841376643773 = (22725985.32612266 - 24402672.47476333) / 22725985.32612266 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: True, SkipValidation: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 24.127571887755103 > 23.733828522727276.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 11/1/2022 10:15:31 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 10:15:31 AM.
IsRegressionStdDev: Marked as regression because -54.51125641752902 (T) = (0 -24344252.53520924) / Math.Sqrt((7938922058.475235 / (19)) + (20059342398.98886 / (33))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (33) - 2, .025) and -0.07725331042656677 = (22598447.644193817 - 24344252.53520924) / 22598447.644193817 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 3.565132374413145 > 3.186866617519949.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 11/1/2022 10:15:31 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 10:15:31 AM.
IsRegressionStdDev: Marked as regression because -80.26210132336969 (T) = (0 -3563380.5664228) / Math.Sqrt((511560300.07826686 / (19)) + (475857533.3813695 / (33))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (33) - 2, .025) and -0.1693566937692946 = (3047299.926027386 - 3563380.5664228) / 3047299.926027386 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: False, SkipValidation: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 3.7025209603729596 > 3.3726054487012993.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 11/1/2022 10:15:31 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 10:15:31 AM.
IsRegressionStdDev: Marked as regression because -29.095599152803345 (T) = (0 -3733771.4978226973) / Math.Sqrt((4632810528.394419 / (19)) + (468973314.41645885 / (33))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (33) - 2, .025) and -0.1430889178485037 = (3266387.6270013344 - 3733771.4978226973) / 3266387.6270013344 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: True, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 24.250224733333337 > 23.56634000291667.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 11/1/2022 10:15:31 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 10:15:31 AM.
IsRegressionStdDev: Marked as regression because -49.773724720145985 (T) = (0 -24142181.699484646) / Math.Sqrt((16987438810.80464 / (19)) + (11366186911.374796 / (33))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (33) - 2, .025) and -0.07823207849632206 = (22390524.434361834 - 24142181.699484646) / 22390524.434361834 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: False, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 3.5541278928571427 > 3.1988951076490517.
IsChangePoint: Marked as a change because one of 8/4/2022 9:24:57 PM, 10/25/2022 8:17:52 AM, 11/1/2022 10:15:31 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 10:15:31 AM.
IsRegressionStdDev: Marked as regression because -66.7244283434966 (T) = (0 -3538603.6454279795) / Math.Sqrt((622064326.354422 / (19)) + (637853905.1536119 / (33))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (33) - 2, .025) and -0.15749309892264632 = (3057127.2076883973 - 3538603.6454279795) / 3057127.2076883973 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 3.65707433627451 > 3.367251755141701.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 11/1/2022 10:15:31 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 10:15:31 AM.
IsRegressionStdDev: Marked as regression because -94.48012761582685 (T) = (0 -3681152.3062214116) / Math.Sqrt((376620662.3032119 / (19)) + (225631100.36416408 / (33))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (33) - 2, .025) and -0.1527646610062979 = (3193325.082509014 - 3681152.3062214116) / 3193325.082509014 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS alpine 3.15
Baseline 5108757b997c59ab8ba1fc5309ab0d4e730e2b77
Compare f4b715f1cfadcbf85486f369724528c051be5733
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Basic

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteBasicUtf16 - Duration of single invocation 509.82 ns 547.59 ns 1.07 0.00 False
WriteBasicUtf16 - Duration of single invocation 549.05 ns 585.78 ns 1.07 0.00 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Basic*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: True, DataSize: 10)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 547.5911681552847 > 535.7870679080074.
IsChangePoint: Marked as a change because one of 7/30/2022 7:04:11 AM, 10/25/2022 8:17:52 AM, 11/1/2022 10:15:31 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 10:15:31 AM.
IsRegressionStdDev: Marked as regression because -46.888481782276486 (T) = (0 -551.36600175533) / Math.Sqrt((10.003887806556584 / (19)) + (7.110681342842126 / (33))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (33) - 2, .025) and -0.07904343010909327 = (510.97665429424455 - 551.36600175533) / 510.97665429424455 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: False, DataSize: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 585.7774028494625 > 576.4293926581229.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 11/1/2022 10:15:31 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 10:15:31 AM.
IsRegressionStdDev: Marked as regression because -35.81839436871512 (T) = (0 -587.6475890990293) / Math.Sqrt((3.849475694981683 / (19)) + (31.2673932794969 / (33))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (33) - 2, .025) and -0.06993843108231802 = (549.2349578513433 - 587.6475890990293) / 549.2349578513433 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 5108757b997c59ab8ba1fc5309ab0d4e730e2b77
Compare f4b715f1cfadcbf85486f369724528c051be5733
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToWriter - Duration of single invocation 494.40 ns 526.79 ns 1.07 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;LargeStructWithProperties&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToWriter


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 526.7944117920856 > 519.1655589264567.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 11/1/2022 10:15:31 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 10:15:31 AM.
IsRegressionStdDev: Marked as regression because -21.764477568808655 (T) = (0 -526.3826136941298) / Math.Sqrt((40.08637700983236 / (19)) + (12.810410501293909 / (33))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (33) - 2, .025) and -0.06991881570068316 = (491.9836963044762 - 526.3826136941298) / 491.9836963044762 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 5108757b997c59ab8ba1fc5309ab0d4e730e2b77
Compare f4b715f1cfadcbf85486f369724528c051be5733
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Strings

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteStringsUtf16 - Duration of single invocation 9.71 ms 10.54 ms 1.09 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Strings*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: False, Escaped: OneEscaped)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 10.544787483766232 > 10.184434033541669.
IsChangePoint: Marked as a change because one of 9/11/2022 6:59:01 PM, 10/25/2022 8:17:52 AM, 11/1/2022 10:15:31 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 10:15:31 AM.
IsRegressionStdDev: Marked as regression because -23.244668971883556 (T) = (0 -10403592.182843966) / Math.Sqrt((7950325571.005873 / (19)) + (13938876109.848766 / (33))) is less than -2.0085591120996775 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (33) - 2, .025) and -0.06927612635383669 = (9729565.56911034 - 10403592.182843966) / 9729565.56911034 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions