proposal: add package for using SIMD instructions

[mpldr]

SIMD has the potential to greatly increase performance of a lot of data processing applications. Since #35307 was closed with the remark

We agree that there is an opportunity here, but we don't know what it looks like. Also, this is an area that is likely to be affected by generics, which are in progress. For this specific proposal, there is no change in consensus. Closing.

Generics are now available, so I want to use this opportunity to necrobump and suggest a simd package, which allows using SIMD instructions via a highlevel API. I think a simple API like Rust's experimental SIMD support, or the previously linked WebAssembly and Dart would help a lot of developers in improving their data processing performance in a simple and easy way.

[mpldr]

As for what the API might look like, the following yenc encoder may provide some insight.

Rust

fn yenc(input: [u8;64]) -> [u8;128] {
    use core::simd::*;
    let mut indata = u8x64::from_array(input); //u8x64::from_array(input);
    indata += u8x64::splat(42);

    // mark special characters
    let mut mask = indata.lanes_eq(u8x64::splat(0));
    mask |= indata.lanes_eq(u8x64::splat(10));
    mask |= indata.lanes_eq(u8x64::splat(13));
    mask |= indata.lanes_eq(u8x64::splat(61));

    indata += mask.select(
        u8x64::splat(64), // add 64 where the mask is set to true
        u8x64::splat(0)   // and don't do anything where it isn't
        );
    
    let escape = mask.select(
        u8x64::splat(61), // add '=' (ASCII 61) where the mask is set to true
        u8x64::splat(0)   // and don't do anything where it isn't
    );
    
    let mut result: [u8;128] = [0;128];
    
    for i in 0..64 {
        result[i*2] = escape[i];
        result[i*2+1] = indata[i];
    }
    
    return result;
}

(Rust Playground)

Go

package simd

This is only supposed to be a very rough draft

package simd

type Uint8x64 [64]uint8

func Add[T SIMD128|SIMD256|SIMD512](x, y T) T {
    // call appropriate add function based on required instruction set
}

func  Uint8x64Splat(value uint8) T {
    // create Uint8x64
}

package simd/mask

This is only supposed to be a very rough draft

package mask

type Maskx64 [64]bool

// 64 lane equality opereator
func Equals64(x, y simd.Uint8x64) *Maskx64  {
    // call appropriate comparison function based on required instruction set
}

func (m1 *Maskx64) Or(m2 *Maskx64) {
    // perform logical or for all lanes
}

func (m *Mask) Select(onTrue, onFalse simd.Uint8x64) simd.Uint8x64 {
    // generate from mask
}

package simd/{sse2,sse3,…}

Specific implementations. Potentially including a fallback if the instruction set is not supported.

Equivalent Go-Code would look something like this (assuming abovementioned API design)

package yenc

import (
    "simd"
    "simd/mask"
)

func yenc(input [64]byte) [128]byte {
    indata := simd.Uint8x64(input)
    indata += Uint8x64Splat(42)

    // mark special characters
    escapeMask := mask.Equals(indata, Uint8x64Splat(0))
    escapeMask.Or(mask.Equals(indata, Uint8x64Splat(10)))
    escapeMask.Or(mask.Equals(indata, Uint8x64Splat(13)))
    escapeMask.Or(mask.Equals(indata, Uint8x64Splat(61)))

    indata += escapeMask.Select(
        Uint8x64Splat(64), // add 64 where the mask is set to true
        Uint8x64Splat(0)   // and don't do anything where it isn't
        )
    
    let escape = escapeMask.Select(
        Uint8x64Splat(61), // add '=' (ASCII 61) where the mask is set to true
        Uint8x64Splat(0)   // and don't do anything where it isn't
    )
    
    var result [128]byte
    
    for i := 0; i < 64; i++ {
        result[i*2] = escape[i]
        result[i*2+1] = indata[i]
    }
    
    return result
}

I think this should be part of the standard library mainly for two reasons:

• This would allow to write performant SIMD code without using assembler within the standard library (making code more readable)
• Lowers the barrier of entry for using SIMD by providing a reliable API

[ianlancetaylor]

The difficulty with a general purpose approach to SIMD, which is what you are suggesting, is that the performance can be dramatically different on different processors. Also, for specific processors, performance is not optimal as not all special purpose instructions are available.

(The difficulty with a processor-specific approach to SIMD is that you have to write different code for each processor.)

(As a side note, I don't see any reason to have a package like simd/sse2 in your description. Instead, we would arrange to use the appropriate implementation when building the simd package.)

[Zheng-Xu]

I hope the proposal can support different architectures. Arm SVE has a feature called (VLA)Vector Length Agnostics. Different H/W may implement the vector size differently. See: a sneak peek into sve and vla programming

On the API level, it is better that we can decide the vector length at runtime instead of hard coding it as 8x64. Reference: OpenJDK Vector API

On the compiler side, so far Go ABI needs the frame size to be decided at compile time, which doesn't support VLA programming very well yet.

[mpldr]

we would arrange to use the appropriate implementation when building the simd package

That might not be as easy as it sounds since that would make binaries potentially less portable. Adding a fallback to use in case a specific instructionset is not supported would help in offsetting this. (Yes, I'm aware that this is also not as simple as it sounds.

[qiulaidongfeng]

(As a side note, I don't see any reason to have a package like simd/sse2 in your description. Instead, we would arrange to use the appropriate implementation when building the simd package.)

I think we should provide different packages for different instruction sets .
Because it helps programmers better design compatible programs for different hardware.
This also helps to maintain compatibility, because the behavior of a specific instruction set depends on the hardware, the package provides an API based on the specific instruction set.

[ianlancetaylor]

@mpldr We already have a mechanism for making binaries more or less portable: the GOAMD64 environment variable and friends. See https://pkg.go.dev/cmd/go#hdr-Environment_variables.

[inkeliz]

I think we should provide different packages for different instruction sets .

It's not the same of writing assembly file for each CPU? I think that defeats the purpose of SIMD package.

---

I think the Zig approach is better (and I think Rust is similar). For instance, Zig provides one @Vector function which works on any CPU, can read more about that here. That makes the code as portable as non-simd version, and it will use SIMD-instructions when available.

[mpldr]

I think the Zig approach is better (and I think Rust is similar). For instance, Zig [provides one ***@***.***`](https://ziglang.org/documentation/master/#Vectors) function which works on any CPU, can [read more about that here](https://zig.news/michalz/fast-multi-platform-simd-math-library-in-zig-2adn). That makes the code as portable as non-simd version.

I also like the approach of just allowing the various operations to be applied to it, but this would require a language change. (Also adding 60-something new types to the language seems rather contrary to "the Go spirit")

[qiulaidongfeng]

I think the Zig approach is better (and I think Rust is similar). For instance, Zig provides one @Vector function which works on any CPU, can read more about that here. That makes the code as portable as non-simd version, and it will use SIMD-instructions when available.

This seems to make the code more portable, and I support it.

[beoran]

Indeed, a vectorize package that is portable and that uses the CPU relevant instructions would be the best solution. On platforms that do not have such instructions, the default implementation could still be useful for portability and optimized for performance.

[qiulaidongfeng]

The current consensus on portability is to use appropriate implementations when building SIMD packages, which I support.

[rsc]

I came across https://github.com/google/highway a few weeks ago. Is there anything we can learn from that about portable API for SIMD?

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group