SIMD Vector128 Atomic Write - Just a Curiosity Question on why it Requires a Pointer

[toupswork]

I'm not going to be so arrogant as to assume the .NET team didn't have a good reason for this API design. I'm just curious what that reason is, because I suspect it's interesting. 🤓

TL;DR

When storing a Vector128<T>, why does StoreAligned(T* destination) require a pointer, while the APIs that accept a managed byref (ref T) do not provide the same aligned-store semantics?

For example, StoreAligned() can result in:

vmovdqa xmmword ptr [rdi], xmm0

whereas an unaligned store may result in:

vmovups xmmword ptr [rdi], xmm0

On hardware where an aligned 128-bit VMOVDQA store is architecturally guaranteed to be atomic, this distinction is important.

Was requiring a pointer for StoreAligned() an intentional way of communicating that the caller is responsible for guaranteeing the required 16-byte alignment?

Or is there some other reason why an aligned store cannot or should not be exposed using a managed byref?

Goal

On x64 hardware where aligned 128-bit stores have the required atomicity guarantee, I'd like to use a SIMD register to publish a 128-bit value without tearing.

One motivating example is a small tagged union.

Simple Example: C-Style Union

Consider:

[StructLayout(LayoutKind.Explicit)] // sizeof(ValueUnion) = 16 bytes
public struct ValueUnion
{
    [FieldOffset(0)]
    public long LongValue;

    [FieldOffset(0)]
    public double DoubleValue;

    [FieldOffset(8)]
    public byte Tag; // 1 = long, 2 = double
}

Logically, the value at offset 0 and the tag at offset 8 form one value.

If those are written using separate memory operations, a concurrent reader could observe an inconsistent combination. For example, it could observe the new payload with the old tag.

What I'd like to do instead is construct all 128 bits in an XMM register and then publish the entire struct with one aligned 128-bit store.

Using a SIMD 128-bit Store

Using Vector128<ulong>, I can construct the complete representation in xmm0 and then use StoreAligned():

[StructLayout(LayoutKind.Explicit)]
public struct ValueUnion
{
    [FieldOffset(0)]
    public ulong LongValue;

    [FieldOffset(0)]
    public double DoubleValue;

    [FieldOffset(8)]
    public byte Tag; // 1 = long, 2 = double
}

unsafe class Program
{
    static void Main()
    {
        ValueUnion vu = default;

        AtomicWrite(&vu, 10UL);

        Console.WriteLine(vu.LongValue);
        Console.WriteLine(vu.Tag);

        AtomicWrite(&vu, 10.0D);

        Console.WriteLine(vu.DoubleValue);
        Console.WriteLine(vu.Tag);
    }

    [MethodImpl(MethodImplOptions.NoInlining)]
    static void AtomicWrite(ValueUnion* destination, ulong longValue)
    {
        Vector128.Create(longValue, 1UL).StoreAligned((ulong*)destination);
    }

    [MethodImpl(MethodImplOptions.NoInlining)]
    static void AtomicWrite(ValueUnion* destination, double doubleValue)
    {
        Vector128.Create(Unsafe.BitCast<double, ulong>(doubleValue), 2UL).StoreAligned((ulong*)destination);
    }
}

Link to Compiler Explorer: https://godbolt.org/z/WjszTGEs5

For the two AtomicWrite methods, RyuJIT produces:

Program:AtomicWrite(ptr,ulong):
       vmovq    xmm0, rsi
       mov      eax, 1
       vpinsrq  xmm0, xmm0, rax, 1
       vmovdqa  xmmword ptr [rdi], xmm0
       ret

Program:AtomicWrite(ptr,double):
       vmovq    rax, xmm0
       vmovq    xmm0, rax
       mov      eax, 2
       vpinsrq  xmm0, xmm0, rax, 1
       vmovdqa  xmmword ptr [rdi], xmm0
       ret

The interesting part is:

vmovdqa xmmword ptr [rdi], xmm0

Assuming the destination is actually 16-byte aligned, this gives me the aligned 128-bit memory operation I was looking for on hardware that provides the corresponding atomicity guarantee.

However, expressing this currently requires unsafe code because StoreAligned() takes a pointer.

[EgorBo]

However, expressing this currently requires unsafe code because StoreAligned() takes a pointer.

The ref overload would require unsafe too, it's no any safer than the unmanaged pointer variant (in fact, it's actually more unsafe). In .NET 11/12, All Loads/Stores accepting ref in SIMD already annotated as "requires unsafe context to call".

Or is there some other reason why an aligned store cannot or should not be exposed using a managed byref?

GC has pointer-sized alignment, so there is no way you can get 16 byte alignment for ref unless:

1. You pin it using fixed and manually align. Any attempt to align it without pinning may lead to silent change in alignment by GC (it can iterrupt at any point and change the alignment).
2. You allocate a native memory and obtain a managed ref out of it.

Both options involve unsafe code by definition.

    static void Main()
    {
        ValueUnion vu = default;

        AtomicWrite(&vu, 10UL);

I don't see what makes ValueUnion here be always 16b aligned on any platform, so your code is incorrect.

[toupswork]

Thanks, Egor! :)

Yeah, that makes sense. I was definitely over-reading the stack layout from the disassembly.

In my test the local happened to land on a 16-byte boundary, but you're right, that obviously isn't a guarantee I can build the code around.

The GC point is the part I was missing. Even if there were a ref overload, a managed ref doesn't give me a stable 16-byte alignment guarantee unless the storage is pinned, and once I'm doing that I'm already in unsafe territory anyway.

Also good to know the ref SIMD load/store APIs are getting marked as requiring unsafe in .NET 11/12. That makes the API shape make a lot more sense.

So I guess the real pattern, if I actually want to do this, is to use pinned or native storage, make sure I own the 16-byte alignment guarantee, and then use the aligned load/store intrinsics.

One thing I'm still curious about: if I use something like NativeMemory.AlignedAlloc and can guarantee the address stays 16-byte aligned, is relying on the resulting vmovdqa being tear-free considered valid from the runtime/JIT side? Or is that still considered outside the contract of the .NET intrinsic API, even if the CPU ISA guarantees the atomicity?

[EgorBo]

One thing I'm still curious about: if I use something like NativeMemory.AlignedAlloc and can guarantee the address stays 16-byte aligned, is relying on the resulting vmovdqa being tear-free considered valid from the runtime/JIT side?

I am not sure we guarantee atomicity anywhere for it, but in fact it should be atomic (afair, on some older CPUs it wasn't so you may need to check Avx.IsSupported for that). BTW, in fact, on modern cpus I bet you can expect atomicity even from unaligned moves on aligned data, but don't take my words as a guarantee, afair, AMD docs didn't mention that explicitly.