Devirtualization with multiple guesses: JIT #86809

EgorBo · 2023-05-26T19:03:19Z

Contributes to #86769

This PR enables GDV with multiple candidates for JIT. (#86551 did the initial infra and enabled that for NativeAOT)
Example:

public interface IValue
{
    int GetValue();
}

public class MyClass1 : IValue
{
    public int GetValue() => 10;
}

public class MyClass2 : IValue
{
    public int GetValue() => 50;
}

public class MyClass3 : IValue
{
    public int GetValue() => 100;
}

public class Program
{
    public static void Main(string[] args)
    {
        for (int i = 0; i < 200; i++)
        {
            Test(new MyClass1()); // MyClass1 has the highest likelihood
            Test(new MyClass1());
            Test(new MyClass2());
            Test(new MyClass3());
            Thread.Sleep(16);
        }
    }

    [MethodImpl(MethodImplOptions.NoInlining)]
    static int Test(IValue value) => value.GetValue();
}

JIT's codegen for Test:

; Assembly listing for method Program:Test(IValue):int
       sub      rsp, 40
       mov      rax, 0x7FFBDC9C2000
       cmp      qword ptr [rcx], rax   ;; is it MyClass1 (50%) ?
       jne      SHORT G_M7592_IG04
       mov      eax, 10
       jmp      SHORT G_M7592_IG07
G_M7592_IG04:
       mov      rax, 0x7FFBDC9C21C8
       cmp      qword ptr [rcx], rax   ;; is it MyClass2 (25%) ?
       jne      SHORT G_M7592_IG06
       mov      eax, 50
       jmp      SHORT G_M7592_IG07
G_M7592_IG06:
       mov      rax, 0x7FFBDC9C2390
       cmp      qword ptr [rcx], rax   ;; is it MyClass3 (25%) ?
       jne      SHORT G_M7592_IG08
       mov      eax, 100
G_M7592_IG07:
       add      rsp, 40
       ret
G_M7592_IG08:
       mov      r11, 0x7FFBDC0C0048    ;; cold virtual fallback (0%)
       call     [r11]IValue:GetValue():int:this
       jmp      SHORT G_M7592_IG07
; Total bytes of code 88

I will keep the number of candidates to check just 1 (current behavior in Main) for JIT untill I implement all work items in #86769

ghost · 2023-05-26T19:03:30Z

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

This PR enables GDV with multiple candidates for JIT. (did the initial infra and enabled that for NativeAOT)
Example:

public interface IValue
{
    int GetValue();
}

public class MyClass1 : IValue
{
    public int GetValue() => 10;
}

public class MyClass2 : IValue
{
    public int GetValue() => 50;
}

public class MyClass3 : IValue
{
    public int GetValue() => 100;
}

public class Program
{
    public static void Main(string[] args)
    {
        for (int i = 0; i < 200; i++)
        {
            Test(new MyClass1());
            Test(new MyClass1());
            Test(new MyClass2());
            Test(new MyClass3());
            Thread.Sleep(16);
        }
    }

    [MethodImpl(MethodImplOptions.NoInlining)]
    static int Test(IValue value) => value.GetValue();
}

JIT's codegen for Test:

; Assembly listing for method Program:Test(IValue):int
       sub      rsp, 40
       mov      r11, qword ptr [rcx]
       mov      rax, 0x7FFBC9FE9C28 ;; is it MyClass1?
       cmp      r11, rax
       jne      SHORT G_M7592_IG04
       mov      eax, 10
       jmp      SHORT G_M7592_IG09
G_M7592_IG04:  
       mov      rax, 0x7FFBC9FE9DF0 ;; is it MyClass2?
       cmp      r11, rax
       jne      SHORT G_M7592_IG06
       mov      eax, 50
       jmp      SHORT G_M7592_IG09
G_M7592_IG06:  
       mov      rax, 0x7FFBC9FE9FB8 ;; is it MyClass3?
       cmp      r11, rax
       jne      SHORT G_M7592_IG08
       mov      eax, 100
       jmp      SHORT G_M7592_IG09
G_M7592_IG08:  
       mov      r11, 0x7FFBC9400310 ;; virtual fallback
       call     [r11]IValue:GetValue():int:this
G_M7592_IG09:  
       nop      
       add      rsp, 40
       ret      
; Total bytes of code 92

I will keep the number of candidates to check just 1 (current behavior in Main) for JIT untill I implement all work items in #86769

Author:	EgorBo
Assignees:	EgorBo
Labels:	`area-CodeGen-coreclr`
Milestone:	-

EgorBo · 2023-05-26T19:38:06Z

/azp run runtime-coreclr pgo, runtime-coreclr pgostress, runtime-coreclr libraries-pgo

azure-pipelines · 2023-05-26T19:38:28Z

Azure Pipelines successfully started running 3 pipeline(s).

EgorBo · 2023-05-26T21:12:20Z

/azp run runtime-coreclr pgo, runtime-coreclr pgostress

azure-pipelines · 2023-05-26T21:12:39Z

Azure Pipelines successfully started running 2 pipeline(s).

EgorBo · 2023-05-26T21:31:39Z

/azp run runtime-coreclr pgo, runtime-coreclr pgostress

azure-pipelines · 2023-05-26T21:31:54Z

Azure Pipelines successfully started running 2 pipeline(s).

EgorBo · 2023-05-27T00:04:44Z

@AndyAyersMS @jakobbotsch PTAL, passes all the tests. I'm going to revert the default number of candidates back to 1 (and enable multiple candidates for pgostress only) - just wanted to make sure it passes CI tests.

NOTE: I recommend to review with "Hide whitespaces" option enabled.

EgorBo · 2023-05-27T18:56:16Z

/azp run runtime-coreclr pgo, runtime-coreclr pgostress

azure-pipelines · 2023-05-27T18:56:30Z

Azure Pipelines successfully started running 2 pipeline(s).

EgorBo · 2023-05-28T00:37:49Z

Stats for the minimal-api (TodosApp) NativeAOT + Exact classes:
829 calls are expanded with 2 or 3 type checks
+80.5Kb size regression (+0.3% size increase) - doesn't look bad for 26.6Mb app

EgorBo · 2023-05-30T14:26:24Z

PTAL @AndyAyersMS, this also fixed a JitDump-only assert

AndyAyersMS · 2023-05-31T20:14:52Z

src/coreclr/jit/importercalls.cpp

+        }
+        else
+        {
+            // We're allowed to make more than 2 guesses - pick all types with likelihood >= 10%


Note with an 8 entry reservoir the smallest value we'll ever see is 12.5%.

I assume it's an implementation detail JIT doesn't have to know, right? (and we might change it) Do you want me to change it to 12.5?

No, just pointing out that currently you may never see a candidate that will fail this test.

Ultimately, we might want to compute the conditional probability. Say there are two candidates with likelihoods of 90 and 9. If the target is not the first candidate, the conditional probability of it being the second candidate is now 90%. (and if there was a third that was say 0.9, if it was not the first two, it is 90% likely to be that third one).

But to do this, we'd have to have more confidence in those low likelihoods.

Btw, with the current logic, the 10% threshold is likely be ignored for anything more complicated than a simple method because inliner will fine it for too low block weight

AndyAyersMS · 2023-05-31T20:18:30Z

src/coreclr/jit/indirectcalltransformer.cpp

+
+            if (IsChainingSupported())
+            {
+                call->SetIsGuarded();


I don't understand this change

@AndyAyersMS it's a theoretical case. IsGuarded is only used for GDV chaining. Imagine we have:

obj.Call1(); obj.Call2();

and Call1() is expanded using 2 classes and chaining is disabled (since it only works with a single candidate). Call2() is expanded with just 1 class - it will try to "chain" with the previous Call1 that doesn't support chaining.

PS: this PR is zero diffs, it doesn't yet enable multiple GDV

Are you sure? I think IsGuarded is currently just used to annotate the inlining tree and inline XML. It means that a callee was GDV and we now decided to test for it and inline it. So I think you can just call it always here.

The inline tree and XML will get confused when one call site expands to multiple candidates. We should think about how we want to express something like multi-guess GDV for those situations. But that can wait.

You're right, I've reverted that change as it's not needed

AndyAyersMS · 2023-05-31T20:18:39Z

src/coreclr/jit/indirectcalltransformer.cpp

+
+            if (IsChainingSupported())
+            {
+                call->SetIsGuarded();


EgorBo · 2023-05-31T20:30:47Z

src/coreclr/jit/gentree.cpp

+                if ((inlineInfo != nullptr) && (inlineInfo->exactContextHnd != nullptr))
+                {
+                    printf(" (exactContextHnd=0x%p)", dspPtr(inlineInfo->exactContextHnd));
+                }


These changes are JitDump only, gtDispTree was not aware that a call might have more than a single candidate.

…sed to produce unexpected diffs

EgorBo · 2023-06-01T22:10:20Z

Zero diffs: https://dev.azure.com/dnceng-public/public/_build/results?buildId=292939&view=ms.vss-build-web.run-extensions-tab

Implement multiple GDV for JIT

aff9029

EgorBo marked this pull request as draft May 26, 2023 19:03

dotnet-issue-labeler bot added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label May 26, 2023

ghost assigned EgorBo May 26, 2023

EgorBo mentioned this pull request May 26, 2023

GDV: Multiple type-checks work items #86769

Open

14 tasks

Clean up

862a23b

Clean up

7ed025b

EgorBo marked this pull request as ready for review May 26, 2023 21:26

Clean up

ce787ad

EgorBo requested review from jakobbotsch and AndyAyersMS May 27, 2023 00:02

EgorBo added 2 commits May 27, 2023 20:43

Merge branch 'main' of github.com:dotnet/runtime into multiple-gdv-jit

a815495

Fix asserts

03bc8f4

build-analysis bot mentioned this pull request May 27, 2023

Could not load file or assembly 'Microsoft.CodeAnalysis.NetAnalyzers #84995

Closed

Update jitconfigvalues.h

91fc31d

AndyAyersMS reviewed May 31, 2023

View reviewed changes

EgorBo commented May 31, 2023

View reviewed changes

Merge branch 'main' of github.com:dotnet/runtime into multiple-gdv-jit

d104307

EgorBo added 2 commits June 1, 2023 14:45

Address feedback

03bda0d

Address feedback

4dfb1b9

AndyAyersMS approved these changes Jun 1, 2023

View reviewed changes

build-analysis bot mentioned this pull request Jun 1, 2023

Tracking issue for CI build timeouts #76454

Closed

Do ClearInlineInfo after we calculate elseBlock's likelihood - it u…

61f61b2

…sed to produce unexpected diffs

EgorBo merged commit 89c96dc into dotnet:main Jun 1, 2023
138 checks passed

EgorBo deleted the multiple-gdv-jit branch June 1, 2023 22:36

EgorBo mentioned this pull request Jun 8, 2023

[Perf] Linux/arm64: 3 Improvements on 6/2/2023 3:25:13 AM dotnet/perf-autofiling-issues#18608

Closed

dotnet locked as resolved and limited conversation to collaborators Jul 2, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Devirtualization with multiple guesses: JIT #86809

Devirtualization with multiple guesses: JIT #86809

EgorBo commented May 26, 2023 •

edited

ghost commented May 26, 2023

EgorBo commented May 26, 2023

azure-pipelines bot commented May 26, 2023

EgorBo commented May 26, 2023

azure-pipelines bot commented May 26, 2023

EgorBo commented May 26, 2023

azure-pipelines bot commented May 26, 2023

EgorBo commented May 27, 2023 •

edited

EgorBo commented May 27, 2023

azure-pipelines bot commented May 27, 2023

EgorBo commented May 28, 2023 •

edited

EgorBo commented May 30, 2023

AndyAyersMS May 31, 2023

EgorBo May 31, 2023 •

edited

AndyAyersMS May 31, 2023

EgorBo May 31, 2023

AndyAyersMS May 31, 2023

EgorBo May 31, 2023 •

edited

AndyAyersMS May 31, 2023

EgorBo Jun 1, 2023

AndyAyersMS May 31, 2023

EgorBo May 31, 2023

EgorBo commented Jun 1, 2023

Devirtualization with multiple guesses: JIT #86809

Devirtualization with multiple guesses: JIT #86809

Conversation

EgorBo commented May 26, 2023 • edited

ghost commented May 26, 2023

EgorBo commented May 26, 2023

azure-pipelines bot commented May 26, 2023

EgorBo commented May 26, 2023

azure-pipelines bot commented May 26, 2023

EgorBo commented May 26, 2023

azure-pipelines bot commented May 26, 2023

EgorBo commented May 27, 2023 • edited

EgorBo commented May 27, 2023

azure-pipelines bot commented May 27, 2023

EgorBo commented May 28, 2023 • edited

EgorBo commented May 30, 2023

Choose a reason for hiding this comment

EgorBo May 31, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

EgorBo May 31, 2023 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

EgorBo commented Jun 1, 2023

EgorBo commented May 26, 2023 •

edited

EgorBo commented May 27, 2023 •

edited

EgorBo commented May 28, 2023 •

edited

EgorBo May 31, 2023 •

edited

EgorBo May 31, 2023 •

edited