JIT: Eagerly invalidate GC state for registers trashed by linker relaxation sequences

[jakobbotsch]

TLS accesses are optimized in NativeAOT with the help of the linker. The JIT emits a sequence of instructions that is specially recognized, and the linker then rewrites (relaxes) those instructions to a more efficient pattern.

Normally, when the JIT emits instructions, it only lazily updates GC state. For example, if rax contains a GC reference, we delay any actual GC information update until we actually emit an instruction that clobbers rax. This is regardless of whether or not the value in rax died a long time ago.

This is a problem for these linker relaxation sequences. The linker's instructions may end up clobbering registers earlier than we do in our emitted instructions, and in that case we are reporting a GC register as live while it had been trashed.

Fix the case for linux-x64 and linux-arm64 by eagerly emitting GC state updates for these special patterns.

Fix #128060

x64 diff for DispatchContinuations NativeAOT codegen:

@@ -125,11 +125,11 @@ G_M10894_IG03:        ; bbWeight=1, gcrefRegs=0008 {rbx}, byrefRegs=0000 {}, byr
        test     rax, rax
        jne      SHORT G_M10894_IG06
        data16   
+                            ; gcrRegs -[rax]
        lea      rdi, [(reloc 0x4000000000420b88)]
        data16   
        data16   
        call     <unknown method>
-                            ; gcrRegs -[rax]
        mov      r15, gword ptr [rax]
                             ; gcrRegs +[r15]
        test     r15, r15
@@ -165,12 +165,12 @@ G_M10894_IG06:        ; bbWeight=1, gcrefRegs=0001 {rax}, byrefRegs=0008 {rbx},
        mov      rdi, gword ptr [rax+0x10]
        mov      gword ptr [rbp-0x38], rdi
        data16   
+                            ; gcrRegs -[rax]
        lea      rdi, [(reloc 0x4000000000420b88)]
                             ; gcrRegs -[rdi]
        data16   
        data16   
        call     <unknown method>
-                            ; gcrRegs -[rax]
        mov      r15, gword ptr [rax]
                             ; gcrRegs +[r15]
        test     r15, r15

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[Copilot]

Pull request overview

This PR adjusts CoreCLR JIT GC register-state reporting around NativeAOT TLS-access linker relaxation marker sequences so the emitted GC info doesn’t incorrectly report a GC ref still live in registers that the linker-relaxed sequence may clobber (linux-x64: rax, linux-arm64: x0).

Changes:

• x64: when emitting the INS_data16 marker byte used in the TLS relaxation pattern, eagerly mark RAX as GC-dead at that point.
• arm64: when emitting the TLSGD adrp relocation form, eagerly mark R0/x0 as GC-dead at that point.
• For symmetry/clarity, mirror these updates in CodeGen’s gcInfo state (gcMarkRegSetNpt) for the relevant sequences.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File	Description
src/coreclr/jit/emitxarch.cpp	Eagerly clears GC liveness for RAX when emitting INS_data16 TLS marker bytes.
src/coreclr/jit/emitarm64.cpp	Eagerly clears GC liveness for x0 for TLSGD-related adrp emission.
src/coreclr/jit/codegenxarch.cpp	Mirrors the RAX GC-state clear in CodeGen for TLSGD reloc sequences.
src/coreclr/jit/codegenarmarch.cpp	Mirrors the x0 GC-state clear in CodeGen for the arm64 TLS relaxation sequence.

[jakobbotsch]

cc @dotnet/jit-contrib PTAL @kg

[jakobbotsch]

/ba-g Failure to obtain CI agent