internal/cpu: detect OS support for AVX-512 and CPU support for VAES and VPCLMULQDQ instructions for performance optimization of crypto ciphers

[tpaint]

What version of Go are you using (go version)?

$ go version 1.15.5 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env

What did you do?

What did you expect to see?

Detect OS support for AVX-512 registers, detect CPU support for AVX-512 VAES and VPCLMULQDQ instructions.

What did you see instead?

AVX-512 OS support and VAES crypto instructions are not currently supported in Go. We have developed proposed patches for go v1.15.5 for internal/cpu: check OS support for AVX-512 registers and check cpu registers for presence of VAES and VPCLMULQDQ, set flags accordingly. The patches will be contributed and submitted to the Go Gerrit for review.

References:

1. https://www.tomshardware.com/news/intel-10nm-xeon-ice-lake-sp-sunny-cove-core-architecture
2. https://www.intel.com/content/dam/www/public/us/en/documents/manuals/64-ia-32-architectures-software-developer-instruction-set-reference-manual-325383.pdf

[gopherbot]

Change https://golang.org/cl/271521 mentions this issue: internal/cpu: Add detection for OS support of AVX-512 and detection of CPU support for

[martisch]

I dont think the standard library should use AVX-512 as this can cause whole program and other tenants to regress in performance due to downclocking. I think its fine for tailored high performance applications to explicitly use (e.g. by coder or user explitily opting in) AVX-512 routines directly if that is beneficial. For that internal/cpu is not needed to support AVX-512 and sys/cpu with existing AVX-512 support can be used.

Happy to be proven otherwise but AFAIK AVX-512 isnt always a net win for the efficiency across all processes of a machine on al l supported CPU implementations. And therefore this may make the default to use AVX-512 if supported regress Go performance for some applications and machines.

[bcmills]

(Compare #42726.)

[bcmills]

@martisch

I dont think the standard library should use AVX-512 as this can cause whole program and other tenants to regress in performance due to downclocking.

I maintain that in the cases where that matters, it is the responsibility of the OS — not individual language runtimes — to disable AVX512 support for programs that should not use it.

That said, I'm not sure whether we currently have adequate builder coverage to test both AVX-512 and whatever fallback paths would be used when it is disabled.

[martisch]

The reality is that OSes already do quirks to pretend they dont support AVX-512 to avoid the overhead of storing and restoring registers:
#43089

I dont think its feasible for the OS to be able to determine at runtime if the workload benefits or doesnt benefit AVX-512. Especially because its not really possible to disable usage of AVX-512 once a program is running as it may have set internal state to assume AVX-512 can be used.

[vsivsi]

@martisch To be fair, Darwin doesn't "pretend" to not support AVX512.

It optimizes by disabling (via the XCR0 control bits) saving AVX512 register state for processes/threads that haven't yet used any instructions requiring that state to be preserved.

Doing it this way necessitates using a different method for discovering AVX512 support (that is, not treating the XCR0 bits as immutable), but that is not at all the same as needing to somehow trick the OS into enabling AVX512. The CPUID bits correctly reflect the AVX512 features of the hardware, and Darwin documents multiple methods for confirming that the OS supports those features.

On the general performance of AVX512, the new Ice Lake (10th Gen Core) CPUs from Intel have much less severe down clocking of "heavy" AVX512 instructions compared to the previous Xeon based AVX512 implementations. And I expect that trend to continue in the coming years. For more details, see:

https://travisdowns.github.io/blog/2020/08/19/icl-avx512-freq.html

BTW, I am supportive of this proposal, and more generally, all proposals that enable programmers to easily detect and use new instructions and other CPU features at their discretion. When the golang standard library chooses to exploit such features with new code-paths should be determined by a separate benchmark-driven cost/benefit analysis.

[martisch]

@vsivsi
Fair. I should have phrased it differently. As far as I understand Darwin communicates that it doesnt support ZMM registers via XCR0 in current state. According to Intel manuals (Intel 64 and IA-32 Architectures
Software Developer’s Manual Volume 2 2.6.11.1 State Dependent #UD) that means instructions that use ZMM (and some others) will cause an Undefined Instruction exception when executed. One has to first trigger mechanisms to enable ZMM support like executing an instruction that will cause UD (or using some darwin specific init functions) and make Darwin change XCR0 and enable ZMM support. (Is it actually documented that darwin will always catch the UD and reexecute and under which conditions?)

BTW x/sys/cpu supports AVX512 and programmers can choose to take decisions based on that (and with your changes will make detection on darwin work). This proposal doesnt change that.

Historically until there is a confirmed use case in the standard/library it was standard to not add feature set detection to internal/cpu as those feature booleans arent exported to be read outside the std library users.

When the golang standard library chooses to exploit such features with new code-paths should be determined by a separate benchmark-driven cost/benefit analysis.

That is why I added: "Happy to be proven otherwise but AFAIK AVX-512 isnt always a net win for the efficiency across all processes of a machine on all supported CPU implementations." So there would need to be some analysis if the average go program on the most "used" platforms benefits to a degree that outways the potential loss of performance for others.

[jhwolf-intel]

We, Intel, are working on a paper with data to show the impact of AVX-512 to address these concerns. The Travis Downs blog did get several things right about Ice Lake server, though there are differences from the client. And, as @vsivsi noted, "On the general performance of AVX512, the new Ice Lake (10th Gen Core) CPUs from Intel have much less severe down clocking of "heavy" AVX512 instructions compared to the previous Xeon based AVX512 implementations. And I expect that trend to continue in the coming years."

This is true for Ice Lake servers as well. We'll show the actual data in the white paper for the heavy instructions. The Travis Downs blog also noted: "The Ice Lake i5-1035 CPU exhibits only 100 MHz of licence-based downclock with 1 active core when running 512-bit instructions."

Our ICX servers will show similar behavior and exhibit only up to 200MHz of license based downclocking with all active cores when running AVX512L instructions.

The VAES instructions in use for our crypto PR only use these light instructions, so we do not expect much of an impact. @tpaint will update the PR with the benchstat data as requested shortly too.

If there are still concerns about our crypto and cpuid changes, how about we push these to the X-repo so others can do their own testing/experiments?

[jhwolf-intel]

@tpaint @martisch
Here's the GCM benchstat data - thought it is best to keep it all in this forum than in the CL. Again, let us know if we can move the changes to the X-repo to keep iterating...

The following is a Microbenchmark run on a single core of an early engineering sample of Ice Lake,
our 3rd-gen Intel(r) Xeon(r) Scalable Processor using Go's benchstat. The benchmark is a
block cipher throughput test for AES-GCM that is part of the Go language runtime crypto library.
The figure of merit is MB/s. The A/B comparison here is the AES-GCM benchmark run on Ice lake
A) without VAES and VPCLMULQDQ vs. B) with VAES and VPCLMULQDQ. Lower is better for the first set;
higher is better for the second set. Note, that we are still working on improvements to it and will re-post
in the coming weeks to include AESGCMSign8K as well.

name old time/op new time/op delta
AESGCMSeal1K-96 205ns ± 0% 186ns ± 0% -9.08% (p=0.000 n=9+9)
AESGCMOpen1K-96 194ns ± 0% 186ns ± 0% -4.04% (p=0.000 n=9+10)
AESGCMSeal8K-96 1.23µs ± 0% 0.83µs ± 0% -33.01% (p=0.000 n=10+9)
AESGCMOpen8K-96 1.21µs ± 1% 0.87µs ± 0% -28.08% (p=0.000 n=9+10)

name old time/op new time/op delta
AESGCMSeal1K-96 5.00GB/s ± 0% 5.50GB/s ± 0% +10.01% (p=0.000 n=10+9)
AESGCMOpen1K-96 5.28GB/s ± 0% 5.50GB/s ± 0% +4.20% (p=0.000 n=9+10)
AESGCMSeal8K-96 6.64GB/s ± 0% 9.92GB/s ± 0% +49.26% (p=0.000 n=10+9)
AESGCMOpen8K-96 6.78GB/s ± 1% 9.43GB/s ± 0% +39.07% (p=0.000 n=9+10)

Configuration: 1-node 2x pre-production 24-core 3rd-gen Intel(r) Xeon(r) Scalable Processor on Intel reference
platform with 256 GB total memory, HT on, Turbo on.

[martisch]

cc @randall77 @aclements

[martisch]

As AVX512 came up in https://go-review.googlesource.com/c/go/+/379394 internal/cpu,internal/bytealg: add SIMD prefix match for Index/amd64:

Before using AVX512 in runtime/std I think we should

• have an understanding and differentiate between allowing AVX512 that can and that cant cause downclocking and thereby performance regressions for other code
• have builders that specifically run on AVX512 hardware (AMD and newer Intel desktop cpus dont support it) to test it
• consider if the maintenance burden added is worth it, especially since support in OSes doesnt seem very tested and can cause subtle bugs with signals/preemption: runtime: AVX512 register state clobbered by signal on macOS #49233

[dbussink]

For VAES & VPCLMULQDQ, they also add support to working with 256 YMM registers from AVX2. So AVX512 is not a hard prerequisite for using VAES & VPCLMULQDQ.

This also shows in the AMD Ryzen Zen 3 CPUs, which don't support AVX512 but do support VAES & VPCLMULQDQ which can be used on 256 bit registers there.

Implementing that would not incur any of the downsides I think of AVX512 and the potential throttling effects it has, but theoretically would still provide a significant speedup of AES operations. It also avoids issues on Darwin with ZMM registers in that case.

[mvdan]

Another reason to favor AVX2 over AVX512 is that, in practice, many common environments will start building with GOAMD64=v3, but few will use v4 given that it's just not cost-efficient given that it's not widely supported. For example, Arch Linux will build its packages for v3, Fedora are considering similar options, and I imagine other operating systems might already be doing the same.

This is not to say that we cannot detect AVX512 at run-time and use it, but it does mean that detecting AVX2 seems like a better tradeoff - it's more widely available, and the check could happen at compile time more often.