Skip to content

Regressions in Utf8/16 #77551

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture arm64
OS Windows 10.0.25094
Baseline 9290f85d9cd4e6dd523f11f0c3fef7a2b7e15ee4
Compare ecf677d2a596b537a6b237e51a65c5a718a4dbc7
Diff Diff

Regressions in System.Text.Json.Tests.Perf_Deep

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteDeepUtf16 - Duration of single invocation 7.26 ms 9.17 ms 1.26 0.57 False
WriteDeepUtf8 - Duration of single invocation 7.15 ms 7.70 ms 1.08 0.61 False

graph
graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Deep*'

Related Issues

Regressions

Improvements

Payloads

Baseline
Compare

Histogram

Edge Detector Info

Collection Data

System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: False)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 9.171853384615387 > 7.538575350000001.
IsChangePoint: Marked as a change because one of 10/25/2022 2:56:59 AM, 10/27/2022 12:34:03 AM falls between 10/18/2022 9:26:37 AM and 10/27/2022 12:34:03 AM.
IsRegressionStdDev: Marked as regression because -4.536067139047587 (T) = (0 -8398730.973199025) / Math.Sqrt((465211152579.85406 / (30)) + (323591138695.06525 / (7))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (7) - 2, .025) and -0.1549915532249524 = (7271681.727669953 - 8398730.973199025) / 7271681.727669953 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: False, SkipValidation: True)

```log

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 7.69652875 > 7.370063625596817.
IsChangePoint: Marked as a change because one of 10/25/2022 2:56:59 AM, 10/27/2022 12:34:03 AM falls between 10/18/2022 9:26:37 AM and 10/27/2022 12:34:03 AM.
IsRegressionStdDev: Marked as regression because -4.760059050293494 (T) = (0 -7984151.078308934) / Math.Sqrt((277194159250.0971 / (30)) + (111536603338.1855 / (7))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (7) - 2, .025) and -0.10447495158258867 = (7228910.956168396 - 7984151.078308934) / 7228910.956168396 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions