v4 - Update SIMD usage to use Vector256<T>, Vector128<T>, and Vector64<T> where possible.

[JimBobSquarePants]

Motivation

Our SIMD API usage is complicated and platform specific with a lack of support for ARM64 in most areas. With .NET7 and .NET8 additional APIs have been added to the System.Numerics namespace which allow the generalization of many of our existing API with the added benefit of implicit support for ARM64 in addition to our Avx/Sse coverage.

For example, Vector128.Shuffle can be used to replace our existing shuffle methods.

Implementation

We should break the task down into multiple smaller tasks, for example:

• General Intrinsics
• Jpeg colorspace intrinsics
• Png Filter Intrinsics
• etc...

We're definitely going to need a lot of assistance here making the port as the APIs are new.

@SixLabors/core would love to hear your thoughts here. Members of the public are invited to comment also.

[tannergooding]

Please feel free to utilize me for reviews/suggestions!

I may not have all the time, but I'll help where I can

[JimBobSquarePants]

Legend thanks. I'm hoping to snipe a certain Discord channel to see if I can get any takers too!

[wuyu8512]

Maybe using Vector<T> directly will save coding time?
He seems to automatically use Vector256, Vector128...

[JimBobSquarePants]

The method coverage for the abstractions is not the same. We'd need to use the explicit structs.

[tannergooding]

Right, while Vector<T> has some advantages and we try to provide method parity between it and the fixed-length vector types (Vector64/128/256/512<T>), there are ultimately differences and they cannot be drop in replacements for eachother.

Due to being variable length, Vector<T> cannot expose APIs like ExtractMostSignificantBits as you could have, theoretically, something like a 128 elements backing it. Such a large vector would need to return an Int128 to represent a bit per element, and that would pessimize smaller cases and be insufficient for larger element counts. Likewise, it can't be used when you do need a fixed element count, such as for accelerating a Matrix4x4

There are plans to address some of this, by exposing new APIs like IndexOfFirstMatch, IndexOfLastMatch, GetMatchCount (names are not final by any means) to replace the most common usages of ExtractMostSignificantBits. But it won't handle everything. It also won't handle the issue that you can no longer trivially accelerate 1-7 element data sets when Vector<T> contains 8 elements.

To that end, we're looking at providing an ISimdVector<TSelf, T> interface that all the types (Vector<T>, Vector64/128/256/512<T>) will implement. This will provide an API surface more similar to Vector<T> (so parity across all the types, but excluding the APIs that only make sense for fixed-length types). It will allow you to write algorithms such as:

public static float Sum<T>(ReadOnlySpan<T> x)
    where T : INumber<T>
{
    if (Vector512.IsHardwareAccelerated && Vector512<T>.IsSupported)
    {
        return SumSimd<Vector512<T>>(x);
    }
    else if (Vector256.IsHardwareAccelerated && Vector256<T>.IsSupported)
    {
        return SumSimd<Vector256<T>>(x);
    }
    else if (Vector128.IsHardwareAccelerated && Vector128<T>.IsSupported)
    {
        return SumSimd<Vector128<T>>(x);
    }
    else
    {
        return SumScalar(x);
    }

    static float SumScalar(ReadOnlySpan<T> x)
    {
        T sum = T.Zero;

        for (int i = 0; i < x.Length; i++)
        {
            sum += x[i];
        }

        return sum;
    }
    
    static float SumSimd<TVector>(ReadOnlySpan<T> x)
        where TVector : ISimdVector<TVector, T>
    {
        TVector vsum = TVector.Zero;

        ref T xRef = MemoryMarshal.GetReference(x);
        int lastBlockIndex = source.Length - (source.Length % Vector<int>.Count);

        for (int i = 0; i < lastBlockIndex; i += TVector.Count)
        {
            vsum += TVector.LoadUnsafe(ref xRef, i);
        }

        return vsum.Sum() + SumScalar(x.Slice(lastBlockIndex));
    }
}

(noting, I didn't test this code and it's not necessarily optimal; it's just meant to give a simple example of how code sharing can work in the future).