You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This is a data-gathering bug to collect instances in which a GOAMD64 environment variable would be helpful. Please feel free to add to it. My hope is that over time, if/when these use cases accumulate, we will get a clearer picture of whether to add GOAMD64, and if so, what shape it should take.
non-destructive floating point math (X2 = X1 + X0, instead of X1 += X0)
faster duffcopy and OpMove via AVX
some scalar instructions like ANDN (and-not)
sane versions of Ctz, without undefined results on 0 argument
removing CPUID checks from intrinsified math functions (mostly rounding)
shifts/rotates that don't clobber flags and allow
AVX512-specific fp-to int64 conversions
extra AVX512 registers
There is also a problem with opting into 512-bit version of AVX512.
By default only 256-bit version should be used to avoid frequency drop, but there should be a mechanism to opt-in for 512-bit version for faster memmove/crypto/etc
Reacted by Josh Bleecher Snyder, futurevessel and Oleg Kovalov
I thought that the Go runtime and standard library already had ways to check for certain hardware features at run-time. How do these enhancements compare to that? If these were to be made at compile-time, would the run-time checks be removed?
It is preferable to use runtime detection when possible. For any sizeable chunk of code, that's what we currently do. But especially for individual instructions, the overhead for runtime checking can be large. In that case, it can be preferable to "bake in" the decision during compilation. That's what such an environment variable would do.
Apologues for asking more questions - if we add an environment variable, wouldn't it be simpler and faster to simply use the optimal code for the hardware, getting rid of all run-time detection?
Unless the point is to do a best-effort combination of the both, so that very expensive code like crypto would still perform well on newer hardware even if the environment variable wasn't set up properly?
We want to use the environment variable as little as possible, as it bakes the optimization decision in at compile time. As a consequence, the binary cannot then adapt at runtime; it would fail on an old machine, or not use optimal code on a new machine.
This will lead to people having to distribute multiple copies of their binary, possibly many depending on how many options are available.
It's also extra API surface to the compiler which we would have to spec & maintain.
So I think we'd need a really good reason to go down this road.
Reacted by Bryan C. Mills, Josh Bleecher Snyder, Sebastien Binet and Oleg Kovalov
function multi-versioning was proposed some time ago as a compromise between less run-time checks and portability between older/newer CPUs.
I can't find discussion right now but the approach with checking CPUID features once and then using
specialized version looks good (at least on paper).
I have no specific proposals, more over, I'm not familiar with referenced technique well.
This is just a note that there is another approaches between "full compile-time" and "full run-time" selection. Thank you.
Reacted by Bryan C. Mills, Oleg Kovalov and Carlo Alberto Ferraris
For some cases indirect call would be slower than call + branch inside it though. Needs a lot of investigation.
There is also JIT-like approach where we re-write function code segment during startup if there are more optimized versions available. This is not possible for every imaginable platforms, but given this is mostly about AMD64, seems doable in that context.
Reacted by Carlo Alberto Ferraris and Oleg Kovalov
Great ! I have been wanting this for a long time - https://groups.google.com/forum/#!topic/golang-nuts/b75xYuORaok :) My thinking was that using the non-destructive 3 operand form of VEX encoded instructions will alleviate register pressure and will make the binaries much smaller (Unfortunately I know nothing about compiler internals to do a PoC and confirm this).
On binary size, I am seeing a slight decrease 🎉. I tested with 3 production applications. 2 network services and 1 webapp. This is without -ldflags="-s -w". Baseline was with current master.
Service1 - 17488784 to 17482928 = 5856
Service2 - 16163230 to 16151311 = 11919
Webapp - 21802366 to 21794461= 7905
On the perf side, general application benchmarks do not show noticeable improvements. But if I do micro-benchmarks targeting very specific functions - for eg if I benchmark math.Atan2 which has lot of FMA statements, I can see improvement of 2ns.
I looked into the objdump of the binaries to investigate possible reasons for this (with whatever meagre assembly knowledge I have).
One thing that I noticed is that the V(ADD|SUB|MUL|DIV)SD operations always act on registers, and not on memory operands which they are especially useful for in reducing register pressure.
Benchmarks that I ran Friday, got busy and forgot to post: https://perf.golang.org/search?q=upload:20180611.1
Quiet machine, using Austin's perflock to make things even more sane.
Saw -0.5% geomean time change, one benchmark got worse for unclear reasons.
Binaries tended slightly smaller:
FWIW, heapBitsSetType would be much simpler with a 1-bit bitmap, something we've been thinking about. That may obviate the need for the fancy bit manipulation instructions.
It seems like the three immediately useful targets are "baseline", "v2", and "v3". Baseline is what we support today. v2 and v3 each offer new instructions that could be used for better optimizing current Go code (e.g., POPCNT in v2, BMI1/BMI2 in v3).
I don't see any immediate benefit from supporting v4 today. Go doesn't have native SIMD support, and AVX512 hardware support seems comparatively less common today still. (There are also lots of discussions about AVX512 having adverse system-level effects due to CPU down-throttling, etc.)
So my tentative proposal would be GOAMD64=baseline, GOAMD64=v2, and GOAMD64=v3. I'm open to alternatives to "baseline", but the psABI doesn't actually give this level a version name of its own. (But maybe we could suggest they retroactively name it "v1".)
There have also been suggestions that binaries should check at startup time whether the CPU supports the required feature, so we don't get lots of SIGILL issue reports from users. That seems reasonable to me.
Also, there's been some brainstorming about various ways we could automatically compile code multiple ways and efficiently dispatch to the right code paths at a higher-level to balance between binary size and runtime dispatch overhead. I think that's intriguing, but it's not obvious to me that we have a concrete solution here or anyone with time to investigate/implement that today.
Reacted by Akhil Indurti, Jaap Aarts, Oleg Kovalov, Bryan C. Mills, Daniel Martí and Sanjay Menakuru
Historically we've avoided things like GOAMD64=native. Instead we let the bootstrap process set the default value of variables like GOAMD64 based on the machine on which the bootstrap is run.
Which isn't to say that we can't change that behavior, but I think that should be a separate decision, and one that should apply to all the GOARCH environment variables, not just GOAMD64.
Reacted by Bryan C. Mills, Matthew Dempsky, Daniel Martí and Oleg Kovalov
I submitted #45453 as a proposal for this (GOAMD64 architecture variants), along with related proposals #45454 (build tags for distinguishing architecture variants) and #45455 (new intrinsics included in Haswell's BMI2 instruction set).
Reacted by Akhil Indurti, Daniel Martí and Oleg Kovalov
Activity
TocarIP commentedon May 22, 2018
I'd like to add:
There is also a problem with opting into 512-bit version of AVX512.
By default only 256-bit version should be used to avoid frequency drop, but there should be a mechanism to opt-in for 512-bit version for faster memmove/crypto/etc
mvdan commentedon May 22, 2018
I thought that the Go runtime and standard library already had ways to check for certain hardware features at run-time. How do these enhancements compare to that? If these were to be made at compile-time, would the run-time checks be removed?
randall77 commentedon May 22, 2018
It is preferable to use runtime detection when possible. For any sizeable chunk of code, that's what we currently do. But especially for individual instructions, the overhead for runtime checking can be large. In that case, it can be preferable to "bake in" the decision during compilation. That's what such an environment variable would do.
mvdan commentedon May 22, 2018
Apologues for asking more questions - if we add an environment variable, wouldn't it be simpler and faster to simply use the optimal code for the hardware, getting rid of all run-time detection?
Unless the point is to do a best-effort combination of the both, so that very expensive code like crypto would still perform well on newer hardware even if the environment variable wasn't set up properly?
randall77 commentedon May 22, 2018
We want to use the environment variable as little as possible, as it bakes the optimization decision in at compile time. As a consequence, the binary cannot then adapt at runtime; it would fail on an old machine, or not use optimal code on a new machine.
This will lead to people having to distribute multiple copies of their binary, possibly many depending on how many options are available.
It's also extra API surface to the compiler which we would have to spec & maintain.
So I think we'd need a really good reason to go down this road.
quasilyte commentedon May 22, 2018
function multi-versioning was proposed some time ago as a compromise between less run-time checks and portability between older/newer CPUs.
I can't find discussion right now but the approach with checking CPUID features once and then using
specialized version looks good (at least on paper).
I have no specific proposals, more over, I'm not familiar with referenced technique well.
This is just a note that there is another approaches between "full compile-time" and "full run-time" selection. Thank you.
quasilyte commentedon May 22, 2018
For some cases indirect call would be slower than call + branch inside it though. Needs a lot of investigation.
There is also JIT-like approach where we re-write function code segment during startup if there are more optimized versions available. This is not possible for every imaginable platforms, but given this is mostly about AMD64, seems doable in that context.
agnivade commentedon May 23, 2018
Great ! I have been wanting this for a long time - https://groups.google.com/forum/#!topic/golang-nuts/b75xYuORaok :) My thinking was that using the non-destructive 3 operand form of VEX encoded instructions will alleviate register pressure and will make the binaries much smaller (Unfortunately I know nothing about compiler internals to do a PoC and confirm this).
@ianlancetaylor had mentioned -
TocarIP commentedon Jun 7, 2018
I've implemented a prototype in https://go-review.googlesource.com/c/go/+/117295 to see how beneficial this can be.
@agnivade can you check if it with your binaries?
agnivade commentedon Jun 8, 2018
With pleasure :)
agnivade commentedon Jun 10, 2018
I have done some testing.
On binary size, I am seeing a slight decrease 🎉. I tested with 3 production applications. 2 network services and 1 webapp. This is without
-ldflags="-s -w". Baseline was with current master.Service1 - 17488784 to 17482928 = 5856
Service2 - 16163230 to 16151311 = 11919
Webapp - 21802366 to 21794461= 7905
On the perf side, general application benchmarks do not show noticeable improvements. But if I do micro-benchmarks targeting very specific functions - for eg if I benchmark
math.Atan2which has lot of FMA statements, I can see improvement of 2ns.I looked into the
objdumpof the binaries to investigate possible reasons for this (with whatever meagre assembly knowledge I have).One thing that I noticed is that the V(ADD|SUB|MUL|DIV)SD operations always act on registers, and not on memory operands which they are especially useful for in reducing register pressure.
For eg. I see patterns like this throughout -
which I believe can be optimized to -
That's all I could glean. Happy to provide more info if required.
dr2chase commentedon Jun 11, 2018
Benchmarks that I ran Friday, got busy and forgot to post:
https://perf.golang.org/search?q=upload:20180611.1
Quiet machine, using Austin's perflock to make things even more sane.
Saw -0.5% geomean time change, one benchmark got worse for unclear reasons.
Binaries tended slightly smaller:
agnivade commentedon Jun 11, 2018
I apologize for my stupidity. I was comparing with 1.10.2. 😞 I have updated my comment. The results match with @dr2chase's observations.
10 remaining items
randall77 commentedon Mar 8, 2021
FWIW,
heapBitsSetTypewould be much simpler with a 1-bit bitmap, something we've been thinking about. That may obviate the need for the fancy bit manipulation instructions.mdempsky commentedon Mar 17, 2021
I asked Google's C++ toolchain team for advice here, and they pointed out the x86-64 psABI now defines microarchitecture levels with specific feature sets: https://en.wikipedia.org/wiki/X86-64#Microarchitecture_Levels. Also, this blog post from Red Hat giving more context: https://developers.redhat.com/blog/2021/01/05/building-red-hat-enterprise-linux-9-for-the-x86-64-v2-microarchitecture-level/
It seems like the three immediately useful targets are "baseline", "v2", and "v3". Baseline is what we support today. v2 and v3 each offer new instructions that could be used for better optimizing current Go code (e.g., POPCNT in v2, BMI1/BMI2 in v3).
I don't see any immediate benefit from supporting v4 today. Go doesn't have native SIMD support, and AVX512 hardware support seems comparatively less common today still. (There are also lots of discussions about AVX512 having adverse system-level effects due to CPU down-throttling, etc.)
So my tentative proposal would be
GOAMD64=baseline,GOAMD64=v2, andGOAMD64=v3. I'm open to alternatives to "baseline", but the psABI doesn't actually give this level a version name of its own. (But maybe we could suggest they retroactively name it "v1".)There have also been suggestions that binaries should check at startup time whether the CPU supports the required feature, so we don't get lots of SIGILL issue reports from users. That seems reasonable to me.
Also, there's been some brainstorming about various ways we could automatically compile code multiple ways and efficiently dispatch to the right code paths at a higher-level to balance between binary size and runtime dispatch overhead. I think that's intriguing, but it's not obvious to me that we have a concrete solution here or anyone with time to investigate/implement that today.
CAFxX commentedon Mar 22, 2021
How about also
GOAMD64=native, that would pick the highest one available on the machine that is compiling the code?ianlancetaylor commentedon Mar 22, 2021
Historically we've avoided things like
GOAMD64=native. Instead we let the bootstrap process set the default value of variables likeGOAMD64based on the machine on which the bootstrap is run.Which isn't to say that we can't change that behavior, but I think that should be a separate decision, and one that should apply to all the
GOARCHenvironment variables, not justGOAMD64.mdempsky commentedon Apr 9, 2021
I submitted #45453 as a proposal for this (GOAMD64 architecture variants), along with related proposals #45454 (build tags for distinguishing architecture variants) and #45455 (new intrinsics included in Haswell's BMI2 instruction set).
runtime: check a microarchitecture level at startup
runtime: check a microarchitecture level at startup
gopherbot commentedon Sep 21, 2021
Change https://golang.org/cl/351191 mentions this issue:
runtime: check a microarchitecture level at startupmartisch commentedon Aug 12, 2022
Since #45453 is resolved I think we can close this bug too.