-
Notifications
You must be signed in to change notification settings - Fork 5.2k
Closed
Labels
arch-arm64area-System.Text.Jsonos-windowsruntime-coreclrspecific to the CoreCLR runtimespecific to the CoreCLR runtimetenet-performancePerformance related issuePerformance related issuetenet-performance-benchmarksIssue from performance benchmarkIssue from performance benchmark
Description
Run Information
| Architecture | arm64 |
|---|---|
| OS | Windows 10.0.25094 |
| Baseline | 9290f85d9cd4e6dd523f11f0c3fef7a2b7e15ee4 |
| Compare | ecf677d2a596b537a6b237e51a65c5a718a4dbc7 |
| Diff | Diff |
Regressions in System.Text.Json.Tests.Perf_Deep
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| WriteDeepUtf16 - Duration of single invocation | 7.26 ms | 9.17 ms | 1.26 | 0.57 | False | |||||
| WriteDeepUtf8 - Duration of single invocation | 7.15 ms | 7.70 ms | 1.08 | 0.61 | False |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Text.Json.Tests.Perf_Deep*'Related Issues
Regressions
Improvements
Payloads
Histogram
Edge Detector Info
System.Text.Json.Tests.Perf_Deep.WriteDeepUtf16(Formatted: False, SkipValidation: False)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 9.171853384615387 > 7.538575350000001.
IsChangePoint: Marked as a change because one of 10/25/2022 2:56:59 AM, 10/27/2022 12:34:03 AM falls between 10/18/2022 9:26:37 AM and 10/27/2022 12:34:03 AM.
IsRegressionStdDev: Marked as regression because -4.536067139047587 (T) = (0 -8398730.973199025) / Math.Sqrt((465211152579.85406 / (30)) + (323591138695.06525 / (7))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (7) - 2, .025) and -0.1549915532249524 = (7271681.727669953 - 8398730.973199025) / 7271681.727669953 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
```#### System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: False, SkipValidation: True)
```log
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionWindowed: Marked as regression because 7.69652875 > 7.370063625596817.
IsChangePoint: Marked as a change because one of 10/25/2022 2:56:59 AM, 10/27/2022 12:34:03 AM falls between 10/18/2022 9:26:37 AM and 10/27/2022 12:34:03 AM.
IsRegressionStdDev: Marked as regression because -4.760059050293494 (T) = (0 -7984151.078308934) / Math.Sqrt((277194159250.0971 / (30)) + (111536603338.1855 / (7))) is less than -2.0301079282477414 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (30) + (7) - 2, .025) and -0.10447495158258867 = (7228910.956168396 - 7984151.078308934) / 7228910.956168396 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Metadata
Metadata
Assignees
Labels
arch-arm64area-System.Text.Jsonos-windowsruntime-coreclrspecific to the CoreCLR runtimespecific to the CoreCLR runtimetenet-performancePerformance related issuePerformance related issuetenet-performance-benchmarksIssue from performance benchmarkIssue from performance benchmark

