[API Proposal]: Clearer additional method overloads for narrowing/widening a vector

[gerhard17]

Background and motivation

Comming from the need to narrow a Vector256<double> to a Vector128<float>, I was confrontated with three differerent coding possibilities, which produced different code gens (with identical results).

using System.Runtime.Intrinsics;

public static class TestClass
{ 	
	public static Vector128<float> Narrow1(Vector256<double> value) {
		return Vector128.Narrow(value.GetLower(), value.GetUpper());
	}

	public static Vector128<float> Narrow2(Vector256<double> value) {
		return Vector256.Narrow(value, Vector256<double>.Zero).GetLower();
	}
   	
	public static Vector128<float> Narrow3(Vector256<double> value) {
		return Vector256.Narrow(value, value).GetLower();
	}
}

translating to following code on my workstation (Windows x64, NET9.0, AVX2 support)

TestClass.Narrow1(System.Runtime.Intrinsics.Vector256`1<Double>)
    L0000: vmovups ymm0, [rdx]
    L0004: vmovaps ymm1, ymm0
    L0008: vcvtpd2ps xmm1, xmm1
    L000c: vextractf128 xmm0, ymm0, 1
    L0012: vcvtpd2ps xmm0, xmm0
    L0016: vmovlhps xmm0, xmm1, xmm0
    L001a: vmovups [rcx], xmm0
    L001e: mov rax, rcx
    L0021: vzeroupper
    L0024: ret

TestClass.Narrow2(System.Runtime.Intrinsics.Vector256`1<Double>)
    L0000: vcvtpd2ps xmm0, ymmword ptr [rdx]
    L0004: vxorps ymm1, ymm1, ymm1
    L0008: vcvtpd2ps xmm1, ymm1
    L000c: vinsertf128 ymm0, ymm0, xmm1, 1
    L0012: vmovups [rcx], xmm0
    L0016: mov rax, rcx
    L0019: vzeroupper
    L001c: ret

TestClass.Narrow3(System.Runtime.Intrinsics.Vector256`1<Double>)
    L0000: vcvtpd2ps xmm0, ymmword ptr [rdx]
    L0004: vmovaps ymm1, ymm0
    L0008: vinsertf128 ymm0, ymm1, xmm0, 1
    L000e: vmovups [rcx], xmm0
    L0012: mov rax, rcx
    L0015: vzeroupper
    L0018: ret

where Narrow3() seems to be the optimal one on my workstation.

API Proposal

I suggest clearer additional methods for narrowing/widening Vector256<TFrom> to Vector128<TTo>.
Where conversion TFrom/TTo are: double/float, long/int, ulong/uint, int/short, uint/ushort, short/sbyte, ushort/byte.

namespace System.Runtime.Intrinsics;

public class Vector256
{
    public static Vector128<float> Narrow(Vector256<double> value) => ...
    public static Vector128<int> Narrow(Vector256<long> value) => ...
    public static Vector128<uint> Narrow(Vector256<ulong> value) => ...
    public static Vector128<short> Narrow(Vector256<int> value) => ...
    public static Vector128<ushort> Narrow(Vector256<uint> value) => ...
    public static Vector128<sbyte> Narrow(Vector256<short> value) => ...
    public static Vector128<byte> Narrow(Vector256<ushort> value) => ...

    public static Vector256<double> Widen(Vector128<float> value) => ...
    public static Vector256<long> Widen(Vector128<int> value) => ...
    public static Vector256<ulong> Widen(Vector128<uint> value) => ...
    public static Vector256<int> Widen(Vector128<short> value) => ...
    public static Vector256<uint> Widen(Vector128<ushort> value) => ...
    public static Vector256<short> Widen(Vector128<sbyte> value) => ...
    public static Vector256<ushort> Widen(Vector128<byte> value) => ...
}

and analogous methods on Vector128 and Vector512...

Remark: I'm aware that oposed to the current Narrow/Widen methods, these new methods cannot be implemented as generic methods.

API Usage

Vector256<double> v0 = Vector256.Create(1.1, 2.2, 3.3, 4.4);

Vector128<float> v1 = Vector256.Narrow(v0);
Vector256<double> v2 = Vector256.Widen(v1);

Alternative Designs

Clearify the optimal usage in the current API documentation.

Risks

Because of being new overloads, I see no risk.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-runtime-intrinsics
See info in area-owners.md if you want to be subscribed.

[huoyaoyuan]

There are Vector128<float> Avx.ConvertToVector128Single(Vector256<double>) etc on xarch, which is single instruction and more optimal than the capabilities exposed by cross-platform intrinsics.

For cross-platform, the shortest form should be Vector256.Narrow(value, Vector256<double>.Zero).GetLower(), which is still longer since the JIT doesn't infer that the upper bits are unused.

[gerhard17]

You are right! Avx.ConvertToVector128Single() provides even better code gen.
In my opinion that emphasizes the value of this proposal.

using System.Runtime.Intrinsics;
using System.Runtime.Intrinsics.X86;

public static class TestClass
{ 	
	public static Vector128<float> Narrow4(Vector256<double> value) {
		return Avx.ConvertToVector128Single(value);
	}
}

TestClass.Narrow4(System.Runtime.Intrinsics.Vector256`1<Double>)
    L0000: vcvtpd2ps xmm0, ymmword ptr [rdx]
    L0004: vmovups [rcx], xmm0
    L0008: mov rax, rcx
    L000b: vzeroupper
    L000e: ret

[colejohnson66]

Narrow and Widden are too simple. Consider a Vector512<T>.Narrow. Does this produce a Vector64, Vector128, or Vector256? Should be NarrowTo128 or something.

[gerhard17]

I assume that Narrow and Widen are implicit working with half / double bit widths.

So in your example a Vector512<double>.Narrow() would produce a Vector256<float>.

If any ambiguities arise, I would also be fine with an explicit naming. But I think it is not necessary.

[colejohnson66]

It would be weird to have to write Narrow().Narrow() to turn a Vector512 into a Vector128. Hence, my note.

[huoyaoyuan]

Both xarch and arm64 are providing single instruction for narrowing from single 2x wide vector, instead of two 1x wide vectors. Thus, the api shape of Vector128<S> Narrow(Vector256<L>) fits better with the instructions.

A risk would be the containing type of the methods. Should it be contained in the larger or smaller vector type? How does it connect to IsHardwareAccelerated? It would typically require IsHardwareAccelerated of both lengths, since Vector64 isn't accelerated on xarch. It can also introduce challenge for ISimdVector, if ever exposed.

For the naming perspective, while Narrow can be overloaded from parameter types, Widen can produce a conflict with the existing tuple-returning overloads, if the members are put into the type of source vector length. Putting them into destination vector length would cause more confusion, or maybe we can name them as WidenFrom or NarrowFrom.

[gerhard17]

@huoyaoyuan: thanks for your comments!

(1st)

A risk would be the containing type of the methods. Should it be contained in the larger or smaller vector type?

A vector class already uses vector classes of other 'bitness' today.

For example Vector256 is using Vector128 and Vector512:

public static class Vector256
{
  public static Vector256<T> Create<T>(Vector128<T> value);
  public static Vector256<T> Create<T>(Vector128<T> lower, Vector128<T> upper);

  public static Vector128<T> GetLower<T>(this Vector256<T> vector);
  public static Vector128<T> GetUpper<T>(this Vector256<T> vector);
  public static Vector256<T> WithLower<T>(this Vector256<T> vector, Vector128<T> value);
  public static Vector256<T> WithUpper<T>(this Vector256<T> vector, Vector128<T> value);

  public static Vector512<T> ToVector512<T>(this Vector256<T> vector);
  public static Vector512<T> ToVector512Unsafe<T>(this Vector256<T> vector);
}

The Create() methods are implemented in the class corresponding to the type they return.
All extension methods are implemented in the type corresponding to the this argument.

=> Following a similar strategy, my proposal should be changed to:

namespace System.Runtime.Intrinsics;

public class Vector256
{
    public static Vector128<float> Narrow(this Vector256<double> value) => ...
    public static Vector128<int> Narrow(this Vector256<long> value) => ...
    public static Vector128<uint> Narrow(this Vector256<ulong> value) => ...
    public static Vector128<short> Narrow(this Vector256<int> value) => ...
    public static Vector128<ushort> Narrow(this Vector256<uint> value) => ...
    public static Vector128<sbyte> Narrow(this Vector256<short> value) => ...
    public static Vector128<byte> Narrow(this Vector256<ushort> value) => ...

    // All this methods are ambigous now!
    public static Vector512<double> Widen(this Vector256<float>  value) => ...
    public static Vector512<long> Widen(this Vector256<int> value) => ...
    public static Vector512<ulong> Widen(this Vector256<uint> value) => ...
    public static Vector512<int> Widen(this Vector256<short> value) => ...
    public static Vector512<uint> Widen(this Vector256<ushort> value) => ...
    public static Vector512<short> Widen(this Vector256<sbyte> value) => ...
    public static Vector512<ushort> Widen(this Vector256<byte> value) => ...
}

=> But now the Widen methods are ambigous as you suspected.
(for futher conclusions see point 4 below)

(2nd)

How does it connect to IsHardwareAccelerated? It would typically require IsHardwareAccelerated of both lengths, since Vector64 isn't accelerated on xarch.

Today a vector class already uses other Vector classes (see analysis in point 1). IsHardwareAccelerated simply returns, whether vectors of the given kind are somehow hardware accellerated or not. The degree of accelleration is not specified.

The API docu says: "Gets a value that indicates whether the vector operations are subject to hardware acceleration through JIT intrinsic support."

There are already existing APIs which do exactly that on xarch (Vector128 is hardware accellerated, while Vector64 is not):

public class Vector128
{
  public static Vector128<uint> Create(Vector64<uint> lower, Vector64<uint> upper);
}

=> My conclusion: IsHardwareAccelerated can be used as today. The proposal needs no adjustment for that.

(3rd)

It can also introduce challenge for ISimdVector, if ever exposed.

As far as I can see, neither Narrow() nor Widen() nor any other type conversion methods are contained in the internal ISimdVector interface.

=> I know too litle about a future public ISimdVector to judge on that.

(4th)

For the naming perspective, while Narrow can be overloaded from parameter types, Widen can produce a conflict with the existing tuple-returning overloads, if the members are put into the type of source vector length. Putting them into destination vector length would cause more confusion, or maybe we can name them as WidenFrom or NarrowFrom.

I agree! Especially when the new methods should be implemented as extension (this) methods. (matching the analyses in point 1)

=> To prevent any abiguities, we can either mangle the generic type of the result vectors in the new method names. (The 'bit' type of the result vector will be implicit known.)

namespace System.Runtime.Intrinsics;

public class Vector256
{
    public static Vector128<float> NarrowToSingle(this Vector256<double> value) => ...
    public static Vector128<int> NarrowToInt32(this Vector256<long> value) => ...
    public static Vector128<uint> NarrowToUInt32(this Vector256<ulong> value) => ...
    public static Vector128<short> NarrowToInt16(this Vector256<int> value) => ...
    public static Vector128<ushort> NarrowToUInt16(this Vector256<uint> value) => ...
    public static Vector128<sbyte> NarrowToSByte(this Vector256<short> value) => ...
    public static Vector128<byte> NarrowToByte(this Vector256<ushort> value) => ...

    public static Vector512<double> WidenToDouble(this Vector256<float> value) => ...
    public static Vector512<long> WidenToInt64(this Vector256<int> value) => ...
    public static Vector512<ulong> WidenToUInt64(this Vector256<uint> value) => ...
    public static Vector512<int> WidenToInt32(this Vector256<short> value) => ...
    public static Vector512<uint> WidenToUInt32(this Vector256<ushort> value) => ...
    public static Vector512<short> WidenToInt16(this Vector256<sbyte> value) => ...
    public static Vector512<ushort> WidenToUInt16(this Vector256<byte> value) => ...
}

Then also a potential

    public static Vector128<short> NarrowToInt16(this Vector512<long> value) => ...

could be implemented. (like suggested by @colejohnson66)

=> Or we can use an uniform name suffix (like your WidenFrom / NarrowFrom suggestion). Which will also have advantages when designing an common interface like ISimdVector.

In that case, I suggest the names NarrowElements() / WidenElements().

namespace System.Runtime.Intrinsics;

public class Vector256
{
    public static Vector128<float> NarrowElements(this Vector256<double> value) => ...
    public static Vector128<int> NarrowElements(this Vector256<long> value) => ...
    public static Vector128<uint> NarrowElements(this Vector256<ulong> value) => ...
    public static Vector128<short> NarrowElements(this Vector256<int> value) => ...
    public static Vector128<ushort> NarrowElements(this Vector256<uint> value) => ...
    public static Vector128<sbyte> NarrowElements(this Vector256<short> value) => ...
    public static Vector128<byte> NarrowElements(this Vector256<ushort> value) => ...

    public static Vector512<double> WidenElements(this Vector256<float> value) => ...
    public static Vector512<long> WidenElements(this Vector256<int> value) => ...
    public static Vector512<ulong> WidenElements(this Vector256<uint> value) => ...
    public static Vector512<int> WidenElements(this Vector256<short> value) => ...
    public static Vector512<uint> WidenElements(this Vector256<ushort> value) => ...
    public static Vector512<short> WidenElements(this Vector256<sbyte> value) => ...
    public static Vector512<ushort> WidenElements(this Vector256<byte> value) => ...
}

The advantages of NarrowElements() / WidenElements() are:

• It has a better readability when used as an extension method

v128 = v256.NarrowElements();

• and the design could potentially be used for other uniform container types like arrays or generic lists of primitive types. (out of scope of this proposal 😆).

[tannergooding]

There's several pit of failures with the APIs that directly return a larger/smaller vector size, particularly given the other vector size may not be accelerated.

I'm not entirely against the idea of exposing them for convenience, but the top post needs to have a complete proposal covering all of Vector64/128/256/512, names that fit in with the existing conventions, and which are not ambiguous with the current Narrow/Widen APIs.

[dotnet-policy-service]

This issue has been marked needs-author-action and may be missing some important information.