Inliner improvements in .NET 9.0

[EgorBo]

Overview

Inliner is one of the most impactful optimizations in terms of output code performance, primarily because it unlocks many other optimizations. Recently, we've started to more actively utilize various generic abstractions, such as Generic Math/GVMs, to unify existing code and eliminate code duplication. For instance, this allowed us to consolidate Formatting and Parsing logic for both UTF16 and UTF8 strings. Also, we've finally enabled Dynamic PGO by default which makes inliner even more aggressive for hot blocks. Consequently, we've begun encountering the inliner’s limitations more frequently. For .NET 9.0, we are planning the following improvements:

Enable inlining for shared generics requiring a runtime lookup

The simplest example is:

bool Test<T>() => Callee<T>();

bool Callee<T>() => typeof(T) == typeof(int);

Currently, Test<_Canon> never inlines Callee<> due to a runtime lookup needed (in fact, it will be eliminated). Another example that it's not allowed to cast Span<T> to ReadOnlySpan<T> efficiently, etc. It should fix issues like:

• JIT: Don't give up on runtime lookup in inliner too early #81432
• Type check in shared generic doesn't eliminate casts to/from confirmed exact type #49614
• EqualityComparer<T>.Default.Equals doesn't devirtualize in a shared generic #10050
• Calls to Interlocked.CompareExchange<T> prevents inlining #8662
• etc. (there were more)

JanK proposed an algorithm for how we can enable it without too much effort: #81432 (comment) and I've already done initial preparations by moving runtime lookup expansion to a late phase in #81635

Re-think Inliner's time budget

Once inliner runs out of its Time Budget it starts to even ignore [AggressiveInlining] attributes which breaks user's assumptions and produces slower code. The idea is to re-think how it's calculated/checked and, possibly, redesign inliner to start from hot blocks first. The beneficiaries are:

1. Library folks sometimes have to re-shuffle code or use NoInlining to make inliner happy - we should improve that experience
2. 1P/3P code should hit less often and gain performance improvements from that

Good examples of the problem are #90090 and #81565

Enable inlining for methods with EH (Exception Handling)

Currently, we unconditionally give up on EH inside the callee even if in fact we'll remove EH blocks as dead code/redundant. Async/await state machine have some good candidates to benefit from this. Also, some C# expression produce try-finally blocks which JIT can eliminate depending on code (lock, foreach). Example:

void Foo()
{
    Bar();
}

[MethodImpl(MethodImplOptions.AggressiveInlining)]
void Bar()
{
    try
    {
        Console.WriteLine("Hello");
    }
    catch
    {
    }
}

Produces:

Inlines into Program:Foo():this:
  [FAILED: has exception handling] Program:Bar():this

Proposed work

• Inliner budget problem
  • Collect data how often we hit it in libraries and 1P/3P apps
  • Consider hot-block first approach for inlining
  • Consider using ML to estimate time it will take to jit a method
• (Q4'23) Enable inlining for shared generics requiring a runtime lookup
  • Basic implementation that just repeats dictionary lookup inside the callee
  • Limit it to cases where it's useful (e.g. only for cases where we eliminate the lookup after inlining)
• [Best effort] Enable inlining for methods with EH

cc @dotnet/jit-contrib @AndyAyersMS @JulieLeeMSFT

[ghost]

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Overview

Inliner is one of the most impactful optimization in terms of ouput code performance, primary because it unlocks many other optimizations. Recently, we've started to more actively utilize various generic abstractions, such as Generic Math/GVMs, to unify existing code and eliminate code duplications. For instance, this allowed us to consolidate Formatting and Parsing logic for both UTF16 and UTF8 strings. Also, we've finally enabled Dynamic PGO by default which makes inliner even more aggressive for hot blocks. Consequently, we've begun encountering the inliner’s limitations more frequently. For .NET 9.0, we are planning the following improvements:

Enable inlining for shared generics requiring a runtime lookup**

The simplest example is:

bool Test<T>() => Callee<T>();

bool Callee<T>() => typeof(T) == typeof(int);

Currently, Test<_Canon> never inlines Callee<> due to a runtime lookup needed (in fact, it will be eliminated). Another example that it's not allowed to case Span<T> to ReadOnlySpan<T> efficiently, etc.. It should fix issues like:

• JIT: Don't give up on runtime lookup in inliner too early #81432
• Type check in shared generic doesn't eliminate casts to/from confirmed exact type #49614
• EqualityComparer<T>.Default.Equals doesn't devirtualize in a shared generic #10050
• Calls to Interlocked.CompareExchange<T> prevents inlining #8662
• etc. (there were more)

JanK proposed an algorithm on how we can enable it with not too much efforts: #81432 (comment) and I've already done initial preparations by moving runtime lookup expansion to a late phase in #81635

Re-think Inliner's time budget

Once inliner runs out of its Time Budget it starts to even ignore [AggressiveInlining] attributes which brakes user's assumptions and producing slower code. The idea is to re-think how it's calculated/checked and, possibly, redesign inliner to start from hot blocks first. The beneficiars are:

1. Library folks sometimes have to re-shuffle code or use NoInlining to make inliner happy - we should improve that experience
2. 1P/3P code should hit less often and gain performance improvements from that

A good examples of the problem are #90090 and #81565

Enable inlining for methods with EH(Exception Handling)

Currently, we unconditionally give up on EH inside the callee even if in fact we'll remove EH blocks as dead code/redundant. Async/await state machine have some good candidates to benefit from this. Also, some C# expression produce try-finally blocks which JIT can eliminate depending on code (lock, foreach). Example:

void Foo()
{
    Bar();
}

[MethodImpl(MethodImplOptions.AggressiveInlining)]
void Bar()
{
    try
    {
        Console.WriteLine("Hello");
    }
    catch
    {
    }
}

Produces:

Inlines into Program:Foo():this:
  [FAILED: has exception handling] Program:Bar():this

Proposed work

• Inliner budget problem
  • Collect data how often we hit it in libraries and 1P/3P apps
  • Consider hot-block first approach for inlining
  • Consider using ML to estimate time it will take to jit a method
• Enable inlining for shared generics requiring a runtime lookup
  • Basic implementation that just repeates dictionary lookup inside the callee
  • Limit it to cases where it's useful (e.g. only for cases where we eliminate the lookup after inlining)
• [Best effort] Enable inlining for methods with EH

cc @dotnet/jit-contrib @AndyAyersMS @JulieLeeMSFT

Author:	EgorBo
Assignees:	-
Labels:	area-CodeGen-coreclr
Milestone:	-