[Perf] Windows/x64: 4 Improvements on 3/29/2023 3:17:08 PM

[performanceautofiler]

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.18362
Queue	TigerWindows
Baseline	80fe2a1a488e641ec99f6ffd7610561ea501ce83
Compare	3f5cb1805ea040e0869c1da8b8a7476a223ce4d2
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryGetValueTrue<String, String>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
FrozenDictionary - Duration of single invocation	13.96 μs	12.55 μs	0.90	0.23	False
IDictionary - Duration of single invocation	15.51 μs	13.42 μs	0.87	0.44	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueTrue<String, String>*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueTrue<String, String>.FrozenDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.547582168737058 < 13.314494838086748.
IsChangePoint: Marked as a change because one of 3/29/2023 12:02:44 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsImprovementStdDev: Marked as improvement because 9.566330299444886 (T) = (0 -12482.703935774649) / Math.Sqrt((711542.0138335463 / (43)) + (14084.911975657786 / (29))) is greater than 1.994437111770645 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (43) + (29) - 2, .975) and 0.09092517411979657 = (13731.217255618518 - 12482.703935774649) / 13731.217255618518 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

System.Collections.TryGetValueTrue<String, String>.IDictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.419948787625419 < 14.76157318057616.
IsChangePoint: Marked as a change because one of 3/29/2023 1:05:10 AM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsImprovementStdDev: Marked as improvement because 13.183651616831055 (T) = (0 -13460.146705309626) / Math.Sqrt((1535570.3473163212 / (299)) + (164050.80555746646 / (32))) is greater than 1.9672006833568965 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (32) - 2, .975) and 0.09026561534472079 = (14795.68864532916 - 13460.146705309626) / 14795.68864532916 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.18362
Queue	TigerWindows
Baseline	80fe2a1a488e641ec99f6ffd7610561ea501ce83
Compare	3f5cb1805ea040e0869c1da8b8a7476a223ce4d2
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in System.Memory.Span<Char>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
StartsWith - Duration of single invocation	13.40 ns	12.15 ns	0.91	0.11	False

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span<Char>*'

Payloads

Baseline
Compare

Histogram

System.Memory.Span<Char>.StartsWith(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 12.149871774134517 < 12.721491435987184.
IsChangePoint: Marked as a change because one of 1/27/2023 11:43:56 PM, 2/13/2023 6:54:14 PM, 3/6/2023 11:14:52 AM, 3/20/2023 11:25:43 AM, 3/25/2023 12:18:49 AM, 3/29/2023 12:02:44 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsImprovementStdDev: Marked as improvement because 17.39265610978622 (T) = (0 -12.18261827571038) / Math.Sqrt((0.35301990382090226 / (299)) + (0.021711031434226025 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.059008381868537775 = (12.946574699466021 - 12.18261827571038) / 12.946574699466021 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

---

Run Information

Name	Value
Architecture	x64
OS	Windows 10.0.18362
Queue	TigerWindows
Baseline	80fe2a1a488e641ec99f6ffd7610561ea501ce83
Compare	3f5cb1805ea040e0869c1da8b8a7476a223ce4d2
Diff	Diff
Configs	CompilationMode:tiered, RunKind:micro

Improvements in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
EndsWith - Duration of single invocation	3.40 ns	1.44 ns	0.42	0.03	True				Trace	Trace

Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Histogram

Microsoft.Extensions.Primitives.StringSegmentBenchmark.EndsWith

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.442303540301357 < 3.227582139860905.
IsChangePoint: Marked as a change because one of 3/29/2023 12:02:44 PM, 4/3/2023 9:14:01 PM falls between 3/26/2023 7:47:31 AM and 4/3/2023 9:14:01 PM.
IsImprovementStdDev: Marked as improvement because 59.95578961950945 (T) = (0 -1.5005772866479137) / Math.Sqrt((0.010799217950103565 / (299)) + (0.03146258956199577 / (29))) is greater than 1.9672675222597202 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (29) - 2, .975) and 0.5722423892863822 = (3.5080083885463456 - 1.5005772866479137) / 3.5080083885463456 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Baseline JIT Disasm

Complete output

Compare JIT Disasm

Complete output

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[kunalspathak]

dotnet/runtime#83945