Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

System.Security.Cryptography.Tests.Perf_Rfc2898DeriveBytes.DeriveBytes regressed on Unix-like OSes #66830

Closed
adamsitnik opened this issue Mar 18, 2022 · 4 comments
Closed

System.Security.Cryptography.Tests.Perf_Rfc2898DeriveBytes.DeriveBytes regressed on Unix-like OSes #66830

adamsitnik opened this issue Mar 18, 2022 · 4 comments
Labels
area-System.Security tenet-performance Performance related issue

Comments

@adamsitnik
Copy link
Member

System.Security.Cryptography.Tests.Perf_Rfc2898DeriveBytes.DeriveBytes benchmark have regressed by 20-30% for all Unix-like OSes. Windows seems to be unaffected.

commit diff

Ubuntu:

image

Windows:

image

Repro:

git clone https://github.com/dotnet/performance.git
python3 ./performance/scripts/benchmarks_ci.py -f net6.0 net7.0 --filter 'System.Security.Cryptography.Tests.Perf_Rfc2898DeriveBytes.DeriveBytes'
Result Ratio Operating System Bit Processor Name
Same 0.90 Windows 11 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores
Same 0.97 Windows 11 X64 AMD Ryzen 9 5900X
Slower 0.83 Windows 11 X64 Intel Core i5-4300U CPU 1.90GHz (Haswell)
Same 1.00 Windows 10 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same 0.99 Windows 11 X64 Intel Core i9-9900T CPU 2.10GHz
Same 1.04 Windows 11 X64 Unknown processor
Slower 0.78 ubuntu 18.04 X64 AMD Ryzen 9 5900X
Slower 0.71 ubuntu 20.04 X64 AMD Ryzen 9 5900X
Slower 0.82 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Same 1.09 ubuntu 18.04 X64 Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)
Slower 0.75 pop 20.04 X64 Intel Core i7-6600U CPU 2.60GHz (Skylake)
Slower 0.81 alpine 3.13 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Slower 0.81 debian 11 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Slower 0.70 macOS Monterey 12.2.1 Arm64 Apple M1 Max
Same 0.96 Windows 10 Arm64 Microsoft SQ1 3.0 GHz
Same 0.97 Windows 11 Arm64 Microsoft SQ1 3.0 GHz
Same 1.03 Windows 10 X86 Intel Xeon CPU E5-1650 v4 3.60GHz
Same 0.99 Windows 10 X86 Intel Core i7-6700 CPU 3.40GHz (Skylake)
Same 0.99 Windows 10 X86 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same 1.02 Windows 10 Arm Microsoft SQ1 3.0 GHz
Slower 0.69 macOS Big Sur 11.6.3 X64 Intel Core i5-4278U CPU 2.60GHz (Haswell)
Slower 0.73 macOS Monterey 12.2.1 X64 Intel Core i7-4870HQ CPU 2.50GHz (Haswell)
Slower 0.69 macOS Monterey 12.2.1 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)
@ghost
Copy link

ghost commented Mar 18, 2022

Tagging subscribers to this area: @dotnet/area-system-security, @vcsjones
See info in area-owners.md if you want to be subscribed.

Issue Details

System.Security.Cryptography.Tests.Perf_Rfc2898DeriveBytes.DeriveBytes benchmark have regressed by 20-30% for all Unix-like OSes. Windows seems to be unaffected.

commit diff

Ubuntu:

image

Windows:

image

Repro:

git clone https://github.com/dotnet/performance.git
python3 ./performance/scripts/benchmarks_ci.py -f net6.0 net7.0 --filter 'System.Security.Cryptography.Tests.Perf_Rfc2898DeriveBytes.DeriveBytes'
Result Ratio Operating System Bit Processor Name
Same 0.90 Windows 11 X64 AMD Ryzen Threadripper PRO 3945WX 12-Cores
Same 0.97 Windows 11 X64 AMD Ryzen 9 5900X
Slower 0.83 Windows 11 X64 Intel Core i5-4300U CPU 1.90GHz (Haswell)
Same 1.00 Windows 10 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same 0.99 Windows 11 X64 Intel Core i9-9900T CPU 2.10GHz
Same 1.04 Windows 11 X64 Unknown processor
Slower 0.78 ubuntu 18.04 X64 AMD Ryzen 9 5900X
Slower 0.71 ubuntu 20.04 X64 AMD Ryzen 9 5900X
Slower 0.82 ubuntu 18.04 X64 Intel Xeon CPU E5-1650 v4 3.60GHz
Same 1.09 ubuntu 18.04 X64 Intel Core i7-2720QM CPU 2.20GHz (Sandy Bridge)
Slower 0.75 pop 20.04 X64 Intel Core i7-6600U CPU 2.60GHz (Skylake)
Slower 0.81 alpine 3.13 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Slower 0.81 debian 11 X64 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Slower 0.70 macOS Monterey 12.2.1 Arm64 Apple M1 Max
Same 0.96 Windows 10 Arm64 Microsoft SQ1 3.0 GHz
Same 0.97 Windows 11 Arm64 Microsoft SQ1 3.0 GHz
Same 1.03 Windows 10 X86 Intel Xeon CPU E5-1650 v4 3.60GHz
Same 0.99 Windows 10 X86 Intel Core i7-6700 CPU 3.40GHz (Skylake)
Same 0.99 Windows 10 X86 Intel Core i7-7700 CPU 3.60GHz (Kaby Lake)
Same 1.02 Windows 10 Arm Microsoft SQ1 3.0 GHz
Slower 0.69 macOS Big Sur 11.6.3 X64 Intel Core i5-4278U CPU 2.60GHz (Haswell)
Slower 0.73 macOS Monterey 12.2.1 X64 Intel Core i7-4870HQ CPU 2.50GHz (Haswell)
Slower 0.69 macOS Monterey 12.2.1 X64 Intel Core i7-5557U CPU 3.10GHz (Broadwell)
Author: adamsitnik
Assignees: -
Labels:

area-System.Security, tenet-performance
Milestone: -

@vcsjones
Copy link
Member

This looks like the same as #65719. The regression in Unix performance is likely due to the additional error cleanup with OpenSSL.

The regression seems to align with when #65148 was merged.

@vcsjones
Copy link
Member

@bartonjs do we want to even benchmark this? Benchmarking the one shot might be useful, but I'm not so sure about the streaming one.

@adamsitnik
Copy link
Member Author

Apologies for the dup.

@adamsitnik adamsitnik mentioned this issue Mar 18, 2022
Closed
10 tasks
@ghost ghost locked as resolved and limited conversation to collaborators Apr 17, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
area-System.Security tenet-performance Performance related issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@vcsjones @adamsitnik