Skip to content

[Perf] Alpine/x64: 12 Regressions on 10/25/2022 12:27:43 PM #9318

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS alpine 3.15
Baseline 5108757b997c59ab8ba1fc5309ab0d4e730e2b77
Compare f4b715f1cfadcbf85486f369724528c051be5733
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Basic

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteBasicUtf16 - Duration of single invocation 509.82 ns 550.12 ns 1.08 0.00 False
WriteBasicUtf16 - Duration of single invocation 621.26 ns 653.40 ns 1.05 0.00 False
WriteBasicUtf16 - Duration of single invocation 549.05 ns 587.87 ns 1.07 0.00 False

graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Basic*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: True, DataSize: 10)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 550.1172035126156 > 535.7870679080074.
IsChangePoint: Marked as a change because one of 7/30/2022 7:04:11 AM, 10/25/2022 8:17:52 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -46.67318390743765 (T) = (0 -551.5146968619168) / Math.Sqrt((10.003887806556584 / (19)) + (7.06367561163948 / (31))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (31) - 2, .025) and -0.07933443187079259 = (510.97665429424455 - 551.5146968619168) / 510.97665429424455 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: True, SkipValidation: True, DataSize: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 653.3950003065668 > 653.0363814629567.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -25.74083084114781 (T) = (0 -656.8083463537015) / Math.Sqrt((2.0147328685117167 / (19)) + (51.69791008783512 / (31))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (31) - 2, .025) and -0.055068900461402386 = (622.5264966737872 - 656.8083463537015) / 622.5264966737872 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Basic.WriteBasicUtf16(Formatted: False, SkipValidation: False, DataSize: 10)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 587.8693327702701 > 576.4293926581229.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -34.24246398408459 (T) = (0 -587.5633815082118) / Math.Sqrt((3.849475694981683 / (19)) + (32.55877334196741 / (31))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (31) - 2, .025) and -0.06978511310862794 = (549.2349578513433 - 587.5633815082118) / 549.2349578513433 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

### Run Information
Architecture x64
OS alpine 3.15
Baseline 5108757b997c59ab8ba1fc5309ab0d4e730e2b77
Compare f4b715f1cfadcbf85486f369724528c051be5733
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Deep

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteDeepUtf16 - Duration of single invocation 22.60 ms 24.06 ms 1.06 0.00 False
WriteDeepUtf16 - Duration of single invocation 22.60 ms 24.39 ms 1.08 0.00 False
WriteDeepUtf8 - Duration of single invocation 3.35 ms 3.73 ms 1.11 0.06 False
WriteDeepUtf16 - Duration of single invocation 3.22 ms 3.67 ms 1.14 0.00 False
WriteDeepUtf8 - Duration of single invocation 22.69 ms 24.30 ms 1.07 0.00 False
WriteDeepUtf16 - Duration of single invocation 3.03 ms 3.57 ms 1.18 0.00 False
WriteDeepUtf8 - Duration of single invocation 3.04 ms 3.58 ms 1.18 0.06 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Deep*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: True, SkipValidation: True)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 24.059880390476188 > 23.56634000291667.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -49.027307853491756 (T) = (0 -24137092.801931098) / Math.Sqrt((16987438810.80464 / (19)) + (11625683975.086817 / (31))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (31) - 2, .025) and -0.07800479942706817 = (22390524.434361834 - 24137092.801931098) / 22390524.434361834 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: True, SkipValidation: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 24.392714276190475 > 23.733828522727276.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -55.876240121482844 (T) = (0 -24358740.89758613) / Math.Sqrt((7938922058.475235 / (19)) + (17813472249.97746 / (31))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (31) - 2, .025) and -0.07789443244543312 = (22598447.644193817 - 24358740.89758613) / 22598447.644193817 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: False, SkipValidation: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 3.7283577345415786 > 3.3726054487012993.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -29.19997567646703 (T) = (0 -3735565.73006812) / Math.Sqrt((4632810528.394419 / (19)) + (444560781.5508781 / (31))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (31) - 2, .025) and -0.14363821953902903 = (3266387.6270013344 - 3735565.73006812) / 3266387.6270013344 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 3.6683956819747414 > 3.367251755141701.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -94.51256350283069 (T) = (0 -3682309.9935711706) / Math.Sqrt((376620662.3032119 / (19)) + (215313835.12290284 / (31))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (31) - 2, .025) and -0.15312719451599288 = (3193325.082509014 - 3682309.9935711706) / 3193325.082509014 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: True, SkipValidation: False)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 24.29748702857143 > 23.80486186166667.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -58.447831817004655 (T) = (0 -24405963.4398896) / Math.Sqrt((10410309944.617916 / (19)) + (8626087413.98141 / (31))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (31) - 2, .025) and -0.07392322443488808 = (22725985.32612266 - 24405963.4398896) / 22725985.32612266 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 3.572605988405797 > 3.186866617519949.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -81.56415460730825 (T) = (0 -3561501.3120536036) / Math.Sqrt((511560300.07826686 / (19)) + (397402298.40976495 / (31))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (31) - 2, .025) and -0.1687399988541845 = (3047299.926027386 - 3561501.3120536036) / 3047299.926027386 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: False, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 3.578244580612245 > 3.1988951076490517.
IsChangePoint: Marked as a change because one of 8/4/2022 9:24:57 PM, 10/25/2022 8:17:52 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -65.83501702998073 (T) = (0 -3537020.3674765625) / Math.Sqrt((622064326.354422 / (19)) + (632217194.8969885 / (31))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (31) - 2, .025) and -0.15697520161453457 = (3057127.2076883973 - 3537020.3674765625) / 3057127.2076883973 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 5108757b997c59ab8ba1fc5309ab0d4e730e2b77
Compare f4b715f1cfadcbf85486f369724528c051be5733
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Strings

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteStringsUtf16 - Duration of single invocation 9.71 ms 10.49 ms 1.08 0.01 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Strings*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Perf_Strings.WriteStringsUtf16(Formatted: True, SkipValidation: False, Escaped: OneEscaped)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 10.487391541666666 > 10.184434033541669.
IsChangePoint: Marked as a change because one of 9/11/2022 6:59:01 PM, 10/25/2022 8:17:52 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -22.641209827021246 (T) = (0 -10399594.471508106) / Math.Sqrt((7950325571.005873 / (19)) + (14177145552.226562 / (31))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (31) - 2, .025) and -0.06886524353409886 = (9729565.56911034 - 10399594.471508106) / 9729565.56911034 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Architecture x64
OS alpine 3.15
Baseline 5108757b997c59ab8ba1fc5309ab0d4e730e2b77
Compare f4b715f1cfadcbf85486f369724528c051be5733
Diff Diff

Regressions in System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToWriter - Duration of single invocation 494.40 ns 524.15 ns 1.06 0.03 False

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;LargeStructWithProperties&gt;*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Serialization.Tests.WriteJson<LargeStructWithProperties>.SerializeToWriter


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 524.1469543664125 > 519.1655589264567.
IsChangePoint: Marked as a change because one of 10/25/2022 8:17:52 AM, 11/1/2022 2:33:16 AM falls between 10/23/2022 1:14:54 PM and 11/1/2022 2:33:16 AM.
IsRegressionStdDev: Marked as regression because -21.51111910712325 (T) = (0 -526.3168587793095) / Math.Sqrt((40.08637700983236 / (19)) + (13.56612273236627 / (31))) is less than -2.010634757623041 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (31) - 2, .025) and -0.06978516307090267 = (491.9836963044762 - 526.3168587793095) / 491.9836963044762 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions