Synthesize Static PGO for NativeAOT (Delegates)

[EgorBo]

Today, CoreCLR heavily relies on Dynamic PGO to inline delegates even when in theory it doesn't need any profile when the target is known. Since normally NativeAOT doesn't have any profile, it basically almost never inlines delegates. We can workaround it by synthesizing a fake static pgo data during ILC compilation where for every delegate invocation site we try to detect if there is only one (or a few) address-taken methods exist with the same signature, so then ILC can create such a fake static profile and JIT will use its existing infrastructure to perform a GDV-based inlining for delegates.

Example:

[MethodImpl(MethodImplOptions.NoInlining)]
static void Test() => Invoke(() => new Program());

static void Invoke(Func<Program> func) => func(); // only one method matches this delegate invoke

CoreCLR with Dynamic PGO:

G_M24707_IG01:  ;; offset=0x0000
       push     rbx
       sub      rsp, 32
G_M24707_IG02:
       mov      rcx, 0x26622C001F8 
       mov      rax, gword ptr [rcx]
       test     rax, rax
       je       SHORT G_M24707_IG05
G_M24707_IG03:
       mov      rcx, 0x7FFA8B7F0948 
       cmp      qword ptr [rax+0x18], rcx
       jne      SHORT G_M24707_IG06
G_M24707_IG04:
       add      rsp, 32
       pop      rbx
       ret      

G_M24707_IG05:
       <cold section>

(The delegate was inlined and the new Program allocation was optimized out as unused)
PS: Presumably for this specific case we don't even need the GDV check, but that is irrelevant to the issue.

NativeAOT codegen:

G_M24707_IG01:  ;; offset=0x0000
       push     rbx
       sub      rsp, 32
G_M24707_IG02:  ;; offset=0x0005
       mov      rcx, qword ptr [(reloc 0x4000000000420b10)]
       mov      rax, gword ptr [rcx+0x10]
       test     rax, rax
       jne      SHORT G_M24707_IG04
G_M24707_IG03:  ;; offset=0x0015
       lea      rcx, [(reloc 0x4000000000420b80)]      ; System.Func`1[Program]
       call     CORINFO_HELP_NEWSFAST
       mov      rbx, rax
       mov      rcx, rbx
       lea      rdx, gword ptr [(reloc 0x4000000000420b40)]      ; 'Frozen Program.<>c object'
       call     CORINFO_HELP_READYTORUN_DELEGATE_CTOR
       mov      rcx, qword ptr [(reloc 0x4000000000420b10)]
       lea      rcx, [rcx+0x10]
       mov      rdx, rbx
       call     CORINFO_HELP_ASSIGN_REF
       mov      rax, rbx
G_M24707_IG04:  ;; offset=0x0049
       mov      rcx, gword ptr [rax+0x08]
       call     [rax+0x20]System.Func`1[System.__Canon]:Invoke():System.__Canon:this
       nop      
G_M24707_IG05:  ;; offset=0x0051
       add      rsp, 32
       pop      rbx
       ret 

Not inlined, the non-inlined target allocates since the return value escapes.

cc @dotnet/ilc-contrib

[dotnet-policy-service]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

[dotnet-policy-service]

Tagging subscribers to this area: @agocke, @MichalStrehovsky, @jkotas
See info in area-owners.md if you want to be subscribed.

[EgorBo]

Alternatively, we can just create a new JIT-VM API getExactMethods similar to getExactClasses if we don't want to deal with a fake profile infra - it's a bit messy.

[jkotas]

we try to detect if there is only one (or a few) address-taken methods exist with the same signature

I do not expect that this would work well in cases people care about. It is very common to create same delegate over lambda with the same signature. For example, take the Linq scenario that we have discussed in the internal email thread. All callbacks passed into Enumerable.Where have the same signature.

[EgorBo]

we try to detect if there is only one (or a few) address-taken methods exist with the same signature

I do not expect that this would work well in cases people care about. It is very common to create same delegate over lambda with the same signature. For example, take the Linq scenario that we have discussed in the internal email thread. All callbacks passed into Enumerable.Where have the same signature.

I didn't target that specific case, I presume lambdas are still popular outside of LINQ as well. But even for LINQ, I think it might be possible that certain APIs to be quite unique and ILC can, in theory, make speculative guesses (or even with help of AI? 🙂) what kind of lambda is fed into SelectMany which, say, is used only once in the current app. The benefit of the fake static PGO is that it doesn't have to be precise, JIT will filter out irrelevant (stale) profile and the fallback is always kept.