JIT: `initobj` of a struct through a reinterpreted byref to a Vector256 local emits `vmovq ymm, r64` (illegal instruction) with optimizations, and throws InvalidProgramException at Tier0

[benaadams]

Description

Zero-initialising a 32-byte struct through a byref that the JIT can see is really the address of a Vector256<ulong> local (Unsafe.As<Vector256<ulong>, S>(ref v) = default;) produces wrong code on x64:

• With optimizations (FullOpts, TieredCompilation=0, or AggressiveOptimization) the JIT emits vmovq ymm0, rax. There is no VEX.256 encoding of vmovq, so the CPU raises #UD and the process dies with ExecutionEngineException: Illegal instruction (.NET 10) or a fail-fast InternalError from RhThrowHwEx (.NET 9, which emits vmovd ymm0, rax).
• At Tier0 / MinOpts (the default tiered configuration) the same method fails to compile at all: InvalidProgramException: Common Language Runtime detected an invalid program. The Tier0 disassembly stops right after the first instruction of the block.

The same pattern with a Vector128<ulong> local and a 16-byte struct also throws InvalidProgramException at Tier0; with optimizations it happens to emit a legal vmovq xmm0, rax, which zero-extends and so produces the right value by accident.

Writing the zero as a vector (v = Vector256<ulong>.Zero; or v = default;) compiles to vxorps and works in every mode. Found while writing an A/B harness that reinterprets Vector256<ulong> locals as a four-limb UInt256 (the pattern in the harness was res = default; inside an inlined callee whose out UInt256 res argument was such a reinterpreted local).

Reproduction Steps

Single file console app, net10.0, default project settings (dotnet new console, replace Program.cs, AllowUnsafeBlocks not needed):

using System;
using System.Runtime.CompilerServices;
using System.Runtime.Intrinsics;

struct Limbs
{
    public ulong u0, u1, u2, u3;
}

static class Program
{
    [MethodImpl(MethodImplOptions.NoInlining)]
    static Vector256<ulong> ZeroThroughStruct()
    {
        Unsafe.SkipInit(out Vector256<ulong> r);
        Unsafe.As<Vector256<ulong>, Limbs>(ref r) = default;
        return r;
    }

    static void Main() => Console.WriteLine(ZeroThroughStruct());
}

dotnet run -c Release                                 # Tier0: InvalidProgramException
DOTNET_TieredCompilation=0 dotnet run -c Release      # FullOpts: illegal instruction
DOTNET_TieredCompilation=0 DOTNET_JitDisasm=ZeroThroughStruct dotnet run -c Release   # shows vmovq ymm0, rax

NoInlining is only there so the method shows up under DOTNET_JitDisasm on its own; without it the same code is inlined into Main and fails the same way.

Expected behavior

The method returns <0, 0, 0, 0> in every tier. The store should be emitted as a vector zero (vxorps ymm0, ymm0, ymm0 + vmovups), or, if the JIT prefers to keep the struct view, as scalar/xmm stores, exactly as it already does when AVX is disabled (DOTNET_EnableAVX=0 gives xorps xmm0, xmm0 and two movups, and runs correctly).

Actual behavior

Default configuration (tiering on), .NET 10.0.11:

Unhandled exception. System.InvalidProgramException: Common Language Runtime detected an invalid program.
   at Program.ZeroThroughStruct()
   at Program.Main()

DOTNET_JitDisasm shows the Tier0 compile stopping after the first instruction of the block (listed twice, once per attempt, no Total bytes of code):

; Assembly listing for method Program:ZeroThroughStruct():System.Runtime.Intrinsics.Vector256`1[ulong] (Tier0)
; Emitting BLENDED_CODE for generic X64 + VEX + EVEX on Windows
; Tier0 code
...
G_M000_IG01:
       push     rbp
       sub      rsp, 48
       lea      rbp, [rsp+0x30]
       vxorps   xmm4, xmm4, xmm4
       vmovdqu  ymmword ptr [rbp-0x30], ymm4
       mov      qword ptr [rbp+0x10], rcx
G_M000_IG02:
       xor      eax, eax

DOTNET_TieredCompilation=0, .NET 10.0.11:

Fatal error.
System.ExecutionEngineException: Illegal instruction: Attempted to execute an instruction code not defined by the processor.
   at Program.ZeroThroughStruct()

; Assembly listing for method Program:ZeroThroughStruct():System.Runtime.Intrinsics.Vector256`1[ulong] (FullOpts)
; Emitting BLENDED_CODE for generic X64 + VEX + EVEX on Windows
G_M000_IG02:
       xor      eax, eax
       vmovq    ymm0, rax
       vmovups  ymmword ptr [rcx], ymm0
       mov      rax, rcx
G_M000_IG03:
       vzeroupper
       ret
; Total bytes of code 18

Same result with DOTNET_EnableAVX512=0 and with DOTNET_EnableAVX2=0 (as long as AVX is on, the JIT still moves the Vector256 as one ymm register). With DOTNET_EnableAVX=0 the output is correct.

.NET 9.0.19, same source: Tier0 throws InvalidProgramException: The JIT compiler encountered invalid IL code or an internal limitation.; FullOpts emits vmovd ymm0, rax and the process ends with Process terminated. InternalError from System.Runtime.EH.RhThrowHwEx.

Regression?

Not a regression between 9 and 10: .NET 9.0.19 fails the same two ways (with vmovd instead of vmovq in the optimized code). Not tested on .NET 8.

Known Workarounds

Zero the local as a vector rather than through the reinterpreted struct view: r = Vector256<ulong>.Zero; (or r = default;) compiles to vxorps and works in every tier. Equivalently, keep the local as the struct type and reinterpret it as a Vector256 afterwards. Both were verified with the repro.

Configuration

• .NET 10.0.11 runtime (commit e2f47b0110ed922f21a1522da67279133ce28f32), SDK 10.0.400; also .NET 9.0.19.
• Windows 11 Pro for Workstations, 10.0.26220.
• x64, AMD Ryzen 9 9950X (AVX-512 capable). Reproduces with DOTNET_EnableAVX512=0 and DOTNET_EnableAVX2=0; does not reproduce with DOTNET_EnableAVX=0, so it needs a VEX-encoded 32-byte move.
• Not tested on other OSes or on ARM64.

Other information

The IL is ldloca r; call Unsafe.As<Vector256<ulong>, Limbs>; initobj Limbs. After the Unsafe.As is folded the JIT is looking at a block zero-init of a 32-byte struct whose destination is a TYP_SIMD32 local. The optimized codegen looks like the init was retyped into a store of an integer zero constant into the SIMD32 local and then lowered with the GPR-to-XMM move (movd/movq) sized by the destination register, which is only valid for 4- and 8-byte destinations; the Tier0 failure is presumably the same retyped tree tripping a check in the un-optimized path. The Vector128 variant taking the same route explains why it "works" there: vmovq xmm, r64 zero-extends to 128 bits.

Related pattern that may be the same root cause but is not yet minimised: in a larger method, an inlined callee that first wrote such a reinterpreted local with a 32-byte default and then stored two individual 8-byte fields into it caused the FullOpts compile to abort after codegen (truncated JitDisasm listing) and fall back to MinOpts (visible with DOTNET_JitDisasmSummary=1); the field stores on their own compile fine.

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[tannergooding]

Looking and working on a fix, it is indeed bad codegen and we're emitting C4E1FD6EC0

---

Now with that being said, I am a bit interested in what the less simplified repro here looks like because this is in general a bit of an anti-pattern.

In general doing this type of thing will block optimizations and lead to overall worse codegen. You should prefer Unsafe.BitCast for converting between types of the same size like this and typically only at boundaries after constructed.

So this would be an example of something that is trivially better:

Limbs limbs = default;
return Unsafe.BitCast<Limbs, Vector256<ulong>>(limbs);

-or-

return Unsafe.BitCast<Vector256<ulong>, Limbs>(Vector256<ulong>.Zero);

or one of the many variations, particularly as code gets more complex and real world. -- I'd also note that in many cases, particularly for default initialization the JIT is already doing the optimal thing here and no unsafe or special logic is required at all.

[benaadams]

Thanks for the quick look.

The unsimplified shape was a one-process A/B benchmark harness for Nethermind.Int256.UInt256 (a struct of four ulong fields). It loads three builds of the library under different assembly names, so there are three distinct UInt256 types, and drives them from shared operand arrays through a neutral storage type. I used Vector256<ulong> for that storage because the library's real consumers read results as one vector, and reinterpreted at the call:

struct VNew : IVariant
{
    [MethodImpl(MethodImplOptions.AggressiveInlining)]
    public static void Mul(ref Vector256<ulong> a, ref Vector256<ulong> b, out Vector256<ulong> r)
    {
        Unsafe.SkipInit(out r);
        UInt256.Multiply(in Unsafe.As<Vector256<ulong>, UInt256>(ref a),
                         in Unsafe.As<Vector256<ulong>, UInt256>(ref b),
                         out Unsafe.As<Vector256<ulong>, UInt256>(ref r));
    }
}

// hot loop, r is a Vector256<ulong> local:
TV.Mul(ref a[i], ref b[i], out Vector256<ulong> r);
acc ^= r;

Multiply inlines all the way down, and one of its paths does res = default; on the out UInt256 res parameter, which after inlining is the punned local; that is where the vmovq ymm came from. Agreed it is an anti-pattern: the library itself only ever sees real UInt256 locals, and the harness now stores operands in a plain four-ulong struct and reinterprets that instead, which is also the realistic model of the callers.

On Unsafe.BitCast: point taken, and we are switching the library to it at those boundaries (build the Vector256 from the limbs, then res = Unsafe.BitCast<Vector256<ulong>, UInt256>(v) once, and Unsafe.BitCast<UInt256, Vector256<ulong>>(x) for the reads) instead of writing through Unsafe.As byrefs.

One more data point you may find relevant. With the same punned out local, the previous version of the library, which wrote the 64x64 result as a 32-byte res = default followed by two 8-byte field stores (Unsafe.AsRef(in res.u0) = low; Unsafe.AsRef(in res.u1) = high;), made the FullOpts compile of the enclosing loop fail after codegen (the JitDisasm listing stops without Total bytes of code) and fall back to MinOpts, visible in JitDisasmSummary as [MinOpts]; Tier-1 compiled it but left the callee as a call. The field stores on their own compile fine, so the block zero appears to be part of that trigger too. I have not minimised it beyond that; happy to if it would help.

[benaadams]

Follow-up on the Unsafe.BitCast switch, in case the write side is interesting to you.

Converting the reads (Unsafe.As<UInt256, Vector256<ulong>>(ref Unsafe.AsRef(in x)) to Unsafe.BitCast<UInt256, Vector256<ulong>>(x)) was byte-for-byte codegen neutral across every method we checked (Tier-1, AVX-512 and AVX2), so that part is done.

Converting the result stores was not. With

static void Add(in UInt256 a, in UInt256 b, out UInt256 res)
{
    ...
    res = Unsafe.BitCast<Vector256<ulong>, UInt256>(sum);   // was: Unsafe.As<UInt256, Vector256<ulong>>(ref res) = sum;
}

public static UInt256 operator +(in UInt256 a, in UInt256 b)
{
    Add(in a, in b, out UInt256 c);
    return c;
}

operator + (Add inlined, c a local that is returned) went from 108 to 156 bytes: the value is written to a stack temp, reloaded, copied to a second temp, and only then stored to the return buffer, with a 120-byte frame appearing where there was none:

       vpsubq   ymm3, ymm2, ymm1
       vmovups  ymmword ptr [rsp+0x20], ymm3
       vmovdqu  ymm3, ymmword ptr [rsp+0x20]
       vmovdqu  ymmword ptr [rsp+0x58], ymm3
       ...
       vmovdqu  ymm0, ymmword ptr [rsp+0x58]
       vmovdqu  ymmword ptr [rcx], ymm0

operator - grew from 428 to 548 bytes the same way. The standalone Add(in, in, out) was unchanged (the out parameter is memory either way), so it only shows when the out target is a promotable struct local in the inlining caller. We kept the byref store for those nine sites. .NET 10.0.11, x64, UInt256 is a 4 x ulong struct with Vector256.IsHardwareAccelerated true.