[Perf] Windows/x64: 3 Regressions on 4/17/2023 7:48:00 AM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.18362
Queue	TigerWindows
Baseline	7526a4c6a0a095670f1135d983a349b4e867f2e2
Compare	7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Single

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
ToStringWithFormat - Duration of single invocation	208.20 ns	229.04 ns	1.10	0.01	True	1762.0056013194626	1814.9977525593047	1.0300749050968734)	Trace	Trace
ToStringWithFormat - Duration of single invocation	207.99 ns	233.44 ns	1.12	0.00	True				Trace	Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G17")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 229.0369279168859 > 215.04355240660692.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -33.497983028909815 (T) = (0 -230.34037333436078) / Math.Sqrt((2.772058429687184 / (11)) + (10.823913311768566 / (36))) is less than -2.0141033888794695 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (11) + (36) - 2, .025) and -0.12121935055048352 = (205.4373867354955 - 230.34037333436078) / 205.4373867354955 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G17")

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 233.4436118462359 > 217.2048213850995.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -34.004968270836486 (T) = (0 -234.89493018518567) / Math.Sqrt((3.116347124604037 / (12)) + (13.97752444798935 / (36))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (12) + (36) - 2, .025) and -0.13190211336017968 = (207.52230021717466 - 234.89493018518567) / 207.52230021717466 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.18362
Queue	TigerWindows
Baseline	887c043eb94be364188e2b23a87efa214ea57f1e
Compare	7dbf6a5f2fd636db350f9cc31b2dfd3af0eff1a8
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.BinaryWriterExtendedTests

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
WriteAsciiCharArray - Duration of single invocation	496.86 ns	534.77 ns	1.08	0.02	False				Trace	Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.BinaryWriterExtendedTests*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.BinaryWriterExtendedTests.WriteAsciiCharArray(StringLengthInChars: 8000)

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 534.7705285176723 > 519.4786825850663.
IsChangePoint: Marked as a change because one of 3/16/2023 12:15:41 AM, 4/17/2023 1:30:24 AM, 4/25/2023 3:46:39 AM falls between 4/12/2023 8:39:35 PM and 4/25/2023 3:46:39 AM.
IsRegressionStdDev: Marked as regression because -19.87859434862802 (T) = (0 -534.7367400263494) / Math.Sqrt((8.899135126141408 / (13)) + (128.03310516314806 / (35))) is less than -2.0128955989180297 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (35) - 2, .025) and -0.08397325905688745 = (493.31174506241774 - 534.7367400263494) / 493.31174506241774 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Baseline
Compare
Diff

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[kunalspathak]

dotnet/runtime#84587