[Perf] Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

[performanceautofiler]

Run Information

Architecture	x64
OS	Windows 10.0.19042
Baseline	be609c3417923404ef3125426cd5f442ed91fd38
Compare	c8ede5c4b1aba85fc894d5faed42d6d5186eabe1
Diff	Diff

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	2.78 μs	4.94 μs	1.78	0.13	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueFalse<Int32, Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<Int32, Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.941567345707538 > 2.9018606079280462.
IsChangePoint: Marked as a change because one of 12/7/2021 2:32:08 PM, 1/5/2022 4:19:16 PM falls between 12/6/2021 2:46:54 PM and 1/5/2022 4:19:16 PM.
IsRegressionStdDev: Marked as regression because -114.3537974402066 (T) = (0 -4871.091473564835) / Math.Sqrt((4769.861900921713 / (18)) + (12205.738984562271 / (148))) is less than -1.974534575858283 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (148) - 2, .025) and -0.7780990150558209 = (2739.4939383687324 - 4871.091473564835) / 2739.4939383687324 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

category:performance
theme:benchmarks
skill-level:intermediate
cost:medium
impact:small

[dotnet-issue-labeler]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[DrewScoggins]

From this range, but unclear what the cause is. d21a020...8369363

[ghost]

Tagging subscribers to this area: @JulieLeeMSFT
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Run Information

Architecture	x64
OS	Windows 10.0.19042
Baseline	be609c3417923404ef3125426cd5f442ed91fd38
Compare	c8ede5c4b1aba85fc894d5faed42d6d5186eabe1
Diff	Diff

Regressions in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark	Baseline	Test	Test/Base	Test Quality	Edge Detector	Baseline IR	Compare IR	IR Ratio	Baseline ETL	Compare ETL
Dictionary - Duration of single invocation	2.78 μs	4.94 μs	1.78	0.13	True

Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.TryGetValueFalse<Int32, Int32>*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<Int32, Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.941567345707538 > 2.9018606079280462.
IsChangePoint: Marked as a change because one of 12/7/2021 2:32:08 PM, 1/5/2022 4:19:16 PM falls between 12/6/2021 2:46:54 PM and 1/5/2022 4:19:16 PM.
IsRegressionStdDev: Marked as regression because -114.3537974402066 (T) = (0 -4871.091473564835) / Math.Sqrt((4769.861900921713 / (18)) + (12205.738984562271 / (148))) is less than -1.974534575858283 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (148) - 2, .025) and -0.7780990150558209 = (2739.4939383687324 - 4871.091473564835) / 2739.4939383687324 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author:	performanceautofiler[bot]
Assignees:	-
Labels:	os-windows, tenet-performance, tenet-performance-benchmarks, area-CodeGen-coreclr, untriaged, amd-perf
Milestone:	-

[echesakov]

This regression is caused by #61408

|     Method |        Job |                   Toolchain | Size |     Mean |     Error |    StdDev |   Median |      Min |      Max | Ratio | RatioSD | Allocated | Alloc Ratio |
|----------- |----------- |---------------------------- |----- |---------:|----------:|----------:|---------:|---------:|---------:|------:|--------:|----------:|------------:|
| Dictionary | Job-AMXPYD | \base\Core_Root\corerun.exe |  512 | 2.943 us | 0.0488 us | 0.0457 us | 2.935 us | 2.866 us | 3.027 us |  1.00 |    0.00 |         - |          NA |
| Dictionary | Job-JTPDDV | \diff\Core_Root\corerun.exe |  512 | 3.641 us | 0.0484 us | 0.0452 us | 3.625 us | 3.582 us | 3.717 us |  1.24 |    0.02 |         - |          NA |

where base corresponds to 9872424 and diff corresponds to 0ddc132.

cc @EgorBo

[adamsitnik]

I confirm that it's still a thing:

System.Collections.TryGetValueFalse<Int32, Int32>.Dictionary(Size: 512)

Result	Ratio	Operating System	Bit	Processor Name
Slower	0.53	Windows 11	X64	AMD Ryzen Threadripper PRO 3945WX 12-Cores
Slower	0.83	Windows 11	X64	AMD Ryzen 9 5900X
Slower	0.85	Windows 10	X64	Intel Xeon CPU E5-1650 v4 3.60GHz
Faster	1.14	Windows 11	X64	Intel Core i5-4300U CPU 1.90GHz (Haswell)
Slower	0.84	Windows 10	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same	1.07	Windows 11	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Same	0.94	Windows 11	X64	Intel Core i9-9900T CPU 2.10GHz
Slower	0.86	Windows 11	X64	Unknown processor
Same	1.01	Windows 11	X64	Unknown processor
Slower	0.85	ubuntu 20.04	X64	AMD Ryzen 9 5900X
Same	0.98	ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz
Same	0.98	centos 7	X64	Intel Xeon CPU E5530 2.40GHz
Slower	0.61	ubuntu 18.04	X64	Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)
Slower	0.85	alpine 3.13	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Slower	0.87	ubuntu 18.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same	0.92	ubuntu 20.04	X64	Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Slower	0.65	ubuntu 20.04	X64	Intel Core i7-8700 CPU 3.20GHz (Coffee Lake)
Slower	0.87	ubuntu 20.04	Arm64	Unknown processor
Slower	0.86	Windows 10	Arm64	Microsoft SQ1 3.0 GHz
Slower	0.86	Windows 11	Arm64	Microsoft SQ1 3.0 GHz
Slower	0.67	Windows 10	X86	Intel Xeon CPU E5-1650 v4 3.60GHz
Same	0.92	Windows 10	Arm	Microsoft SQ1 3.0 GHz
Slower	0.85	macOS Big Sur 11.6.3	X64	Intel Core i5-4278U CPU 2.60GHz (Haswell)
Faster	1.13	macOS Big Sur 11.4	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)

[EgorBo]

The regression almost got fixed by various PRs so moving it to Future