[Perf -98%] System.Numerics.Tests.Perf_Vector2.GetHashCodeBenchmark

[DrewScoggins]

Run Information

Architecture	x64
OS	Windows 10.0.18362
Changes	diff

Regressions in System.Numerics.Tests.Perf_Vector2

Benchmark	Baseline	Test	Test/Base	Modality	Baseline Outlier
GetHashCodeBenchmark	1.21 ns	2.41 ns	1.98	Bimodal	False

Historical Data in Reporting System

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f netcoreapp5.0 --filter 'System.Numerics.Tests.Perf_Vector2*';

Histogram

System.Numerics.Tests.Perf_Vector2.GetHashCodeBenchmark

[0.996 ; 1.247) | @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[1.247 ; 1.498) | 
[1.498 ; 1.749) | 
[1.749 ; 1.975) | 
[1.975 ; 2.226) | @@@@@@@@@@@@@@@@@@@@@@@@@
[2.226 ; 2.408) | @@@@@@@@
[2.408 ; 2.659) | @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[2.659 ; 2.945) | @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
[2.945 ; 3.074) | @
[3.074 ; 3.347) | 
[3.347 ; 3.598) | @@@@
[3.598 ; 3.864) | @

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

[DrewScoggins]

Similar to #39029

[stephentoub]

This is the same as #39029 (comment).

[ghost]

Tagging subscribers to this area: @tannergooding, @pgovind
Notify danmosemsft if you want to be subscribed.

[adamsitnik]

I confirm that this benchmark has regressed on every OS, adding the 5.0 milestone

System.Numerics.Tests.Perf_Vector2.GetHashCodeBenchmark

Conclusion	Base	Diff	Base/Diff	Modality	Operating System	Bit	Processor Name	Base Runtime	Diff Runtime
Slower	0.96	3.89	0.25		Windows 10.0.18363.1016	Arm	Microsoft SQ1 3.0 GHz	.NET Core 3.1.6	5.0.100-rc.1.20413.9
Slower	1.21	2.57	0.47		Windows 10.0.18363.959	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	.NET Core 3.1.6	5.0.100-rc.1.20404.3
Slower	1.03	2.33	0.44		Windows 10.0.18363.959	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	.NET Core 3.1.6	5.0.100-rc.1.20418.3
Slower	1.19	3.00	0.40		Windows 10.0.19041.450	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)	.NET Core 3.1.6	5.0.100-rc.1.20413.9
Slower	1.02	2.55	0.40		ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	.NET Core 3.1.6	5.0.100-rc.1.20403.23
Slower	1.18	2.58	0.46		macOS Mojave 10.14.5	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)	.NET Core 3.1.6	5.0.100-rc.1.20404.2

[adamsitnik]

Vector3 has regressed as well, but I am not creating a new issue:

System.Numerics.Tests.Perf_Vector3.GetHashCodeBenchmark

Conclusion	Base	Diff	Base/Diff	Modality	Operating System	Bit	Processor Name	Base Runtime	Diff Runtime
Slower	1.55	5.97	0.26		Windows 10.0.18363.1016	Arm	Microsoft SQ1 3.0 GHz	.NET Core 3.1.6	5.0.100-rc.1.20413.9
Slower	1.69	3.02	0.56		Windows 10.0.18363.959	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	.NET Core 3.1.6	5.0.100-rc.1.20404.3
Slower	1.90	2.83	0.67		Windows 10.0.18363.959	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	.NET Core 3.1.6	5.0.100-rc.1.20418.3
Slower	2.21	3.54	0.62		Windows 10.0.19041.450	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)	.NET Core 3.1.6	5.0.100-rc.1.20413.9
Slower	1.79	5.42	0.33		ubuntu 18.04	X64	Intel Xeon CPU E5-1650 v4 3.60GHz	.NET Core 3.1.6	5.0.100-rc.1.20403.23
Slower	2.06	3.87	0.53		macOS Mojave 10.14.5	X64	Intel Core i7-5557U CPU 3.10GHz (Broadwell)	.NET Core 3.1.6	5.0.100-rc.1.20404.2

[jeffhandley]

Closing this as an expected (and acceptable) regression due to dotnet/corefx#40935.