Composite-r2r with interpreter fallback leads to significant IL_STUB interpretation if FEATURE_JIT is disabled

[BrzVlad]

This behavior was initially observed on ios startup with coreclr enabled by @kotlarmilos, with the overwhelming majority of interpreted methods being IL STUBS on maui sample app ((https://gist.github.com/kotlarmilos/7a6aca4dbeb871c80c3dac4785bf4af6))

In order to reproduce on desktop, I randomly selected the json benchmark (https://github.com/BrzVlad/benchmarks/tree/main/json). I built the runtime with FEATURE_JIT disabled and with added logs for jitted/interpreted methods (BrzVlad@6ee645c). I did a self contained publish of the json project, replaced the runtime binaries and did a composite r2r compilation of the entire app (https://github.com/BrzVlad/benchmarks/blob/main/json/composite.dll.rsp).

If the runtime has jit enabled there is no interpreter compilation of IL_STUBs, but also apparently no fallback to jit either for these stubs. When jit is disabled, roughly half of the methods compiled are IL stubs. https://gist.github.com/BrzVlad/f26bf894b9b8238b830d0750223ecb9b

I'm not sure, but this might suggest that when FEATURE_JIT is enabled, there is an alternative cod generation mechanism for these IL_STUBS that doesn't go through the jit. If that's the case it seems the only solution for ios would be to have a r2r flag that triggers the inclusion of some of these stubs in the image ? Would these stubs be limited enough in numbers for this to be feasible ?

TODO:

• figure out actual impact in terms of startup for these IL stubs on MAUI sample. (rough estimate would be 5-10% of startup)
• figure out why we use so many pinvoke IL stubs, they shouldn't be needed
• investigate scenarios that require the instantiating stubs

[BrzVlad]

@davidwrighton @jkotas @AaronRobinsonMSFT Any thoughts on why this happens and what would be the best solution for this ?

[AaronRobinsonMSFT]

[...] but this might suggest that when FEATURE_JIT is enabled, there is an alternative cod generation mechanism for these IL_STUBS that doesn't go through the jit.

Can we debug this to see where the branch is? On the x86 platform we use the StubLinker to generate code, but that shouldn't apply in this case. I agree with your hypothesis that there must be something else going on here.

what would be the best solution for this ?

I think we need to get to the why before coming up with a solution.

[BrzVlad]

I tested on osx-arm64 with the runtime built with the diff linked in the issue (BrzVlad@6ee645c). I didn't go into more detail with the investigation yet since I suspected this behavior might be expected and also wanted to double check what to look for. Can surely look into it further if the behavior is not by design.

[jkotas]

Any thoughts on why this happens and what would be the best solution for this ?

We have discussed this during the sync. The problem is that we do not have the bespoke shuffle thunk "JIT" when we are not able to generate the code at runtime. We have a few options here:

• Create a special path for shuffle trunks in the interpreter. It may help throughput, but there is still going to be startup overhead.
• Generate code for these IL stubs in R2R. Generating the code itself is easy and it should be very little code. The harder part is how to store the code in R2R image. R2R does not have an infrastructure for storing IL stubs in the R2R images currently. We are building it for runtime async feature. We should make sure that there is a uniform design the covers all (or most) IL stubs.

[BrzVlad]

@jkotas Is there any tracking issue for the R2R IL stubs support, it would be useful for me to keep an eye on.

Some general observations after taking a closer look at the IL stub usage

• by far, the most used IL stubs from the initial report were pinvoke and then instantiating stubs
• instantiating stubs seem simple enough and I believe they would fit well into the functionality provided by the CallStubGenerator (delegate shuffle thunks as well). It might also be awkward for them to be included in the r2r image since we might have to blindly generate a ton of them.
• precompiling pinvoke IL stubs to R2R image seems like a reasonable approach, they also have more IL so it would be more expensive to generate/execute with interpreter at runtime.
• there seems to be a lot of pinvoke IL stubs generated for signatures should require no marshaling so that seems suspicious to me (for example (dynamicClass):IL_STUB_PInvoke(nint,nint)). Future investigation is needed to see why this happens in the maui startup, wasn't able to reproduce locally.

[jkotas]

Is there any tracking issue for the R2R IL stubs support

The infrastructure for R2R IL stubs is being implemented as part of R2R runtime async support (e.g. current active PR #122651).

there seems to be a lot of pinvoke IL stubs generated for signatures should require no marshaling

This does not look right. R2R has support for fully precompiling pinvokes that do not require marshalling. Something does not work as expected here.

We are not able to R2R pinvokes that require marshalling. The fix for that is to switch these pinvokes to LibraryImports so that they do not require marshalling and can be R2R.

instantiating stubs

We may want to check why we need these instantiating stubs and that we are not missing any optimizations. The JIT should avoid the instantiating stubs in many cases, they are really only needed for cases like delegates.

they would fit well into the functionality provided by the CallStubGenerator (delegate shuffle thunks as well)

CallStubGenerator is not a thing on wasm. The problems that we solve with CallStubGenerator will need to be solved again for wasm in a different way. We may want to take it into account when devising solutions for these issues.