internal/cpu: new package to expose processor capabilities

[minux]

We have a lot of packages in the std that uses specialized instructions
that are not always present. Most of the times, the package will detect
the required features by themselves. This is fine but will lead to code
duplication. And as Bryan Chan mentioned on https://golang.org/cl/22201,
even if the processor provides way to detect certain optional features,
it's still better to use AT_HWCAP from Linux auxv because that also
takes kernel support into account.

Only the runtime can access auxv, so it makes sense for the runtime
to query the processor capabilities and provide that to the packages.

I propose that we add an internal package internal/cpu that exposes
capability flags for the current processor so that each std packages
could query it directly instead of having a runtime detection routine
that duplicates the work.

Another benefit is that, some processors, like ARM, doesn't provide
a way to do runtime capability detection, so we have to rely on the
kernel to provide this information. Different kernels provide different
mechanisms for this (sysctl for BSD and auxv for linux), so providing
a package that abstracts those OS-dependent feature away is also
beneficial.

We might promote the package to runtime/cpu if deemed fit, but that's
out of the scope for this proposal.

The package could be modeled after the Linux's AT_HWCAP bits,
and it will be processor dependent.

[mundaym]

I'm happy to work on adding this (particularly the s390x bit).

I'm interested in what thoughts people have for the API. The three options that occur to me are:

1. Function taking strings: cpu.Has("ssse3", "popcnt")
2. Function taking constants: cpu.Has(cpu.SSSE3, cpu.POPCNT))
3. Raw bools: cpu.SSSE3 && cpu.POPCNT)

I'm not sure if we want to prefix with the CPU type to avoid naming conflicts. If so then perhaps s/SSSE3/AMD64.SSSE3/.

It might also be nice if the function (or bools) could be inlined and constant folded. In that case I suspect we'd need to limit the number of features to be checked to one per call. This could be useful in scenarios where a feature is optional on say i386/ppc64, but mandatory on newer versions of the architecture such as amd64/ppc64le.

Another possibility is that we do this in the runtime/internal/sys package. Something like sys.Feature("ssse3") perhaps? There are variables in sys that might be useful to an application as well and could be made part of a public cpu package were we to ever go down that route.

BTW @minux when you say:

it will be processor dependent

Do you mean you want the API to be processor dependent?

[minux]

I mean the constants will be cpu dependent. There is no constant folding though, this package is to check cpu capability at runtime, so I think an API that looks like this is fine: cpu.Has(feats cpu.Feature...) E.g.: cpu.Has(cpu.FloatingPoint, cpu.AES, ...) Yeah, we need to figure out a way to name and category the CPU features. Should each arch be in a separate package (e.g. internal/CPU/arm?) Or prefix each feature with arch name? Do we consolidate common features across arches?

[laboger]

I think prefixing them with the arch name would be good. That seems to be the convention used for many other constants in golang (opcodes, relocations, etc.). That way there would be no confusion in case some cpu features are close but don't mean exactly the same thing on different architectures.

[ceseo]

One idea is to use the same naming conventions that we have in glibc, for consistency (i.e. glibc/sysdeps/powerpc/dl-procinfo.c:_dl_powerpc_cap_flags). This way, the names match the ones shown when LD_SHOW_AUXV=1:

AT_HWCAP: vsx arch_2_06 dfp ic_snoop smt mmu fpu altivec ppc64 ppc32
AT_HWCAP2: tar isel ebb dscr htm arch_2_07

Querying for the capabilities should take into account whether an arch has a HWCAP2 or not, because there is no way to know (a priori) if a capability bit is in HWCAP or HWCAP2. My suggestion here is to use a concatenated 64-bit HWCAP+HWCAP2 mask, so we can easily map the bits to the capabilities. That's what we did in glibc/gcc to implement __builtin_cpu_is() / __builtin_cpu_supports() for Power.

[ceseo]

@minux are you already working on this? Thanks.

[minux]

No, I'm not. Please go ahead if you want to. Thanks.

[ceseo]

@mundaym I think that starting with a runtime/internal/sys approach is a good idea for now. My idea was to create something that would work similarly to __builtin_cpu_supports() in gcc. What do you think?

[mundaym]

I had a play with this last weekend. I ended up getting a bit lost in circular dependencies... You might want to just add (perhaps hackily) what you need directly into the runtime package for now and let it get cleaned up later.

Along those lines it would be easy to add the variables runtime.hwcap (already there for arm) and runtime.hwcap2 and then grab them using assembly in other packages when necessary. Ideally they'd both have the type uint32 and be defined and set in os_linux.go (rather than in an arch-dependent file).

The constants representing features could go in the runtime/internal/sys package but that means they can only be accessed in the runtime package. If you instead put them in a global package like internal/cpu then the runtime package can't get at them. That could be a way to get this proposal started though.

[ceseo]

@mundaym OK. We'll need these checks later, when we start to add the new ISA 3.0 (POWER9) instructions and write runtime optimizations using those.

For now, I was thinking about something trivial, like: ceseo@ca09310

then evolve from that.

[gopherbot]

CL https://golang.org/cl/31149 mentions this issue.