proposal: simd: CPU feature vet check under GOEXPERIMENT=simd

[aclements]

Proposal Details

Background

#73787 proposed a package of architecture-specific SIMD intrinsics, guarded by GOEXPERIMENT=simd for Go 1.26. Because these intrinsics are architecture-specific and tightly tied to underlying CPU instructions, they will panic if the necessary CPU features are not supported by the hardware. As a result, the user of the simd package has to be careful to perform the appropriate CPU feature checks before using operations.

However, the space of CPU features is complex. For example, Intel’s AVX-512 alone has 21 different feature flags. While each intrinsic documents which features it requires, keeping track of this in any non-trivial SIMD code is difficult and error-prone. Furthermore, even with complete test coverage, a missed feature check may not result in a panic in CI if the CI hardware happens to support that feature, and may only panic in production on different hardware.

Proposal

I propose that we add directive comments to mark CPU feature requirements and a vet check that uses these directives to statically verify that the necessary feature checks are performed.

The minimal proposal has just one directive, which can only be applied to a function or method:

//cpu:requires <feature> [&& <feature> ...]
func F(...) { ... }

This directive indicates that the caller of F must ensure all of the listed features are available by calling the appropriate feature check functions before calling F.

The feature names follow the naming convention of the feature check methods in the simd package: for example, X86.AVX or X86.AVX512.

We will annotate all functions and methods in the simd package with feature requirement directives, but the cpu:requires directive is not limited to the simd package.

We will add a “cpu” vet check that verifies all feature annotations are satisfied. It will work by performing a flow analysis of every function G that calls an annotated function F to check that every call to F is reachable only if the necessary feature check functions have returned true. If G itself has a feature annotation, then the flow analysis assumes that all listed features are available on entry to G.

Some CPU features imply other features. For example, a CPU can only support AVX2 if it also supports AVX. The vet check will understand this: if a function requires some feature, the vet check will assume that all implied features are also available within that function.

Function values, interfaces, and constraints

The vet check will ensure that capturing a function H as a function value meets H’s requirements at the point of capture. For example:

//cpu:requires AVX
func H() { ... }

var c1 func()

func I() {
	if !simd.X86.AVX() {
		return
	}
	// H can be captured as a function value here because
	// its requirements are satisfed.
	c1 = H
}

Unlike a function, there is no way to directly annotate the requirements of a function literal. Instead, the vet check will assume that any features guaranteed at the point where the function value is created are available within that function value’s body. For example:

var c2 func()

func J() {
	if !simd.X86.AVX() {
		return
	}
	c2 = func() {
		// This code can assume AVX is available, even though
		// it may be called after J returns.
	}
}

In a sense, the closure also closes over CPU features. This works because the supported features are static during an executable’s lifetime.

Interfaces are trickier. The conservative solution would be to disallow converting a type to an interface if any of the type’s methods have requirements that aren’t satisfied at the point of conversion. It’s not enough to only check the methods matched by the interface because the interface value could be later type-asserted to a wider interface.

Instead, we chose to simply not check interface assignments. We assume these will be rare in SIMD code given its orientation toward performance, and the worst that can happen is that a method panics because a required feature was not checked by the caller. However, see “future directions” for a more complete solution.

While interfaces are fundamentally dynamic, constraints can be checked statically. For a generic instantiation of type T to constraint type C, the vet check assures that any methods of T that satisfy C have their requirements met at the instantiation point.

Compatibility

This proposal is additive only. For now, it would only be in effect when GOEXPERIMENT=simd. The details may change before we standardize the simd package without the GOEXPERIMENT.

Implementation

I (@aclements) already have a prototype implementation of this vet check. The intent would be to land this as part of the Go 1.26 simd experiment.

Future directions

Above is a minimal proposal, but we plan to extend this in a few ways for the final version.

Requirement expressions

We plan to extend the cpu:requires directive to accept boolean expressions using both && and || (but not !). In particular, this will allow functions that have different requirements on different CPU architectures, but it needn’t be limited to that use case.

This complicates the vet check’s analysis somewhat. If G has requirement expression g and calls function F with requirement expression f, then vet must check that g ⇒ f is valid (universally true). Because requirement expressions are monotone formulas (no “not” operator), checking this is relatively straightforward and at worst quadratic in the length of the expression (no SAT solvers required 😀).

Custom feature check functions

In the minimal proposal, there’s no way to write a feature check function outside the simd package. We plan to allow user-written feature check functions by adding an additional directive comment:

//cpu:ensures <feature> [&& <feature>]
func Check() bool { ... }

The feature check function must return bool, and the vet check will ensure that it only returns true if the listed features have actually been checked in its body. We may extend this to functions that panic if the listed features are not available.

Dynamic interface call checks

While there’s no practical solution to statically checking interface method calls, with some help from the compiler we could transform any method requirements into dynamic checks on interface calls. To do this, the compiler would construct wrapper functions for any method with requirements. The wrapper would dynamically check the method’s requirements and produce a clear panic. These wrappers would appear in the interface method tables rather than the underlying methods. This way, direct calls to the method would skip these dynamic checks, while dynamic calls via an interface value would get dynamic checks. This is similar to transformations the compiler already does in other situations involving interfaces.

Feature-based optimization

Some SIMD operations can map to more or less efficient hardware instructions depending on which CPU features are available. While the compiler is able to infer this from explicit CPU feature checks, this is limited to intra-procedural analysis. With cpu:requires directives, it could optimize entire functions under stronger feature assumptions and give programmers more control over these optimizations, while still ensuring inter-procedural safety via the vet check.

[gabyhelp]

Related Issues

• simd: architecture-specific SIMD intrinsics under a GOEXPERIMENT #73787
• proposal: simd: new package for intrinsics #67520 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[robpike]

This opens the door to an ever-expanding world of complexity, incompatibility and run-time surprise. It feels un-Go-like to me.

Ditto - especially- for #73787, which I fear I did not see when it was proposed.

[aclements]

@robpike , can you elaborate?

[ianlancetaylor]

Edited to add: sorry, I somehow missed the comments about interfaces in the proposal.

---

Does this work with interface methods? It seems plausible that somebody might write an interface where different implementations of the interface are selected based on CPU features. The implementation methods could have //cpu directives, but that doesn't help much since those methods generally aren't called directly. The interface methods are called with all ranges of //cpu directives, so the interface itself can't be annotated.

Does this proposal just not help in that scenario?

[ianlancetaylor]

Just to confirm: no restrictions are imposed on the function bodies. There will be no warning if a function with //cpu:requires avx uses a SIMD function that is only available for avx2.

I think that's the right choice but it also means that this doesn't help with what seems to me the most confusing aspect: remembering which functions and types are available for which CPU features.

And that suggests that we could build this up statically with no user written directives. For each function that uses SIMD, vet can in principle compute the set of required features. In other words, it can compute the contents of the //cpu:requires comments. So why do we need the comments?

[cherrymui]

There will be no warning if a function with //cpu:requires avx uses a SIMD function that is only available for avx2.

If I understand correctly, I think there will be. The way it works is that we would mark avx2 operations like Uint64x4.Add with the //cpu:requires avx2 directive, therefore the function that calls Uint64x4.Add must ensure the feature is available before calling it, either by checking the feature inside the function body, or by using the //cpu:requires directive to ensure it is available at entry.

[aclements]

Just to confirm: no restrictions are imposed on the function bodies. There will be no warning if a function with //cpu:requires avx uses a SIMD function that is only available for avx2.

No, the point of the proposed vet check is specifically to warn if one function calls another without ensuring the necessary features. If a function is annotated with //cpu:requires X86.AVX and calls a function that is only available for avx2 (which would be annotated with //cpu:requires X86.AVX2) without dynamically checking for avx2, that will result in a vet warning.

Here are some example to make that more concrete. Suppose these functions are in the simd package:

package simd

//cpu:requires X86.AVX2
func (x Int8x32) Add(y Int8x32) Int8x32

var X86 X86Features
type X86Features struct {}
// (The feature check methods are annotated, too, but that's internal to the vet check.)
func (X86Features) AVX() bool
func (X86Features) AVX2() bool

Given these, here are some examples of using the directive and feature checks. These are the errors from my current vet prototype (hand edited, so there may be minor mistakes):

var x, y simd.Int8x32

//cpu:requires X86.AVX
func F() {
    _ = x.Add(y) // Vet error: call to Int8x32.Add requires AVX2; call simd.X86.AVX2 first or add //cpu:requires X86.AVX2 to F
    if !simd.X86.AVX2() {
        return
    }
    _ = x.Add(y) // Now this is allowed because we did a dynamic check.
}

//cpu:requires X86.AVX
func G() {
    // AVX is assumed to be available on entry, so this can freely call F.
    F() // No vet error
}

// No cpu:requires, so no features are assumed on entry.
func H() {
    G() // Vet error: call to G requires AVX; call simd.X86.AVX first or add //cpu:requires X86.AVX to H
    if !simd.X86.AVX2() {
        return
    }
    // The vet check knows that AVX2 support implies AVX support, so now we can call G.
    G() // No vet error
}

[aclements]

Proposal committee discussion

• We spent a while talking about whether requirements should just be inferred. While technically possible, my main concern is that it's not clear where to report a missing requirement check because it becomes a property of an entire call chain. The function-level annotations are useful to humans, too: they allow the programmer to express their intent, and allow the tooling to pinpoint where the programmer's intent doesn't match the reality of the code.

• What if you only care about machines with, say, AVX2, and don't want to bother annotating lots of functions? One possible solution to this is to analyze feature checks in package init functions. If a init function panics when a feature isn't available, then the analysis can assume that feature is available to all functions in that package. This would at least raise this problem to the package level rather than the function level.

• Instead of a vet check, we could instead have a sanitizer mode that makes it possible to disable certain CPU features and compiles explicit checks into all SIMD operations. This would allow testing environments to simulate CPUs with fewer features than their actual hardware. This is, in a way, a dynamic version of this static check. Unfortunately, the CPU feature space is rather high-dimensional, so this would add a lot of testing time. The static check seems preferable.

• An alternate sanitizer mode we didn't discuss in person: track per goroutine what features have been checked for, and have SIMD intrinsics check that the necessary feature checks have happened. However, since "checked features" would strictly accumulate on a goroutine, this would result in poor "feature coverage." E.g., if F checks for a feature and the test happens to call F before G, it wouldn't get coverage of G's feature checks, even if other code may call G first.

Should the annotation instead be a function call? One advantage of the directive comment is that it makes the requirement clearly part of the function's API (one could always write that anyway as an un-enforceable comments). There are a few options for how to do this:

• For each feature check function, have a parallel "requires" function. That would double the feature check API and provides no way to express "or" requirements.
• Have a designated panic, such that code can use regular control flow, but the vet check still has something to look for. E.g., if simd.X86.AVX2() || simd.ARM.Whatever() { panic(simd.NotSupported) }
• Have a simd.Requires(bool) function that can take an expression of feature checks and signal to the vet check that these features must already be ensured at that point. In the proposal review, we were leaning toward this option.

In all cases, the goal of the vet check would be to ensure that the requirement check never fails. The mechanics of the vet check are similar to the directive comment, except that these checks could in principle occur anywhere in a function. I haven't worked out the details of how that would affect the flow analysis.

If we take the function approach, the natural thing would be to say "this panics if the requirements aren't meant" and the vet check would effectively ensure the panic can never happen. That puts more onus on the compiler to optimize away these checks. With the simd.Requires(bool) function, we could say it's meant as a signal and may or may not panic. That's kind of unsatisfying, but we could make it always panic once we've done the compiler optimization work.

The compiler could optimize these checks (at least at function entry) by treating them as a partial inline into the caller and then "canceling" out the requirement in the the caller, either with an actual dominating feature check or another requirement.

[aclements]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— aclements for the proposal review group