LLVM as a back-end for RyuJIT

[EgorBo]

Since dotnet/corert#8230 thread was archived recently, I'm creating a new one here to continue the discussion.

I'm trying to build a minimal dynamically-loaded extension for RyuJIT to generate LLVM IR (and compile it for tier1 (JIT) or use in AOT scenarios) from GenTrees (HIR/LIR) in order to benefit from RyuJIT's .net specific optimizations and SSA transformation for locals. Something like this:

(my local version is a bit more complicated but it's still just a very high-level prototype and is a mess)

Some of the challenges (too early to make a deep analysis):

• Setup LLVM dependency (currently, I just use a locally built one)
• Make RyuJIT's headers portable, basically, provide an external API, stabilize layout (e.g. a lot of fields here and there depend on different condition symbols)
• Implement all operators, GC/EH, etc - How to draw an owl

Since there are LLILC and #247 efforts I wonder what can be shared/merged. Maybe it makes sense to bind all RyuJIT's structures/API to C# (e.g. via CppAst.NET) and continue IR -> LLVM IR transformation there using LLVMSharp (C# bindings for LLVM C-api). Maybe it's better to join #247 and just port RyuJIT's optimizations one-by-one there.

Initially, I just wanted to make an extension for RyuJIT to replace tier1 with LLVM-JIT for HPC# code (a GC-free subset of C# in Unity's Burst for high-performance stuff) to generate vectorized loops, etc where RyuJIT currently doesn't perform well. But I see that there is a quite a demand for AOT. So maybe you have some thoughts regarding the direction it is better to move to to make it at least theoretically useful?

/cc @jkotas @MichalStrehovsky @yowl @AndyAyersMS

[AndyAyersMS]

Right, EH and GC are still enormous hurdles.

I think you may be able to get pretty far with EH, perhaps only compromising on currently somewhat rare cases -- like filters that have side effects -- that require a true understanding of the order in which things can happen.

I don't believe there is a viable strategy for precise GC with LLVM native codegen right now, especially if you start turning on parts of the LLVM optimizer. That being said, the Azul folks have something; I'm not sure if they've upstreamed the key parts. However, it's likely the story in Java is a bit simpler, as they don't have managed (interior) pointers.

[EgorBo]

I think you may be able to get pretty far with EH, perhaps only compromising on currently somewhat rare cases -- like filters that have side effects -- that require a true understanding of the order in which things can happen.

If I'm not mistaken, in Mono we decided to just IL-rewrite exception filters to normal catchs because of that 🙂

I have no plans for precise GC at the moment as it's definitely too complicated for me.

[jkotas]

Yes, EH and GC are problems that require work. I would not worry about them initially. GC can be conservative first and EH filters are rare.

Random thoughts:

• Trying to build a transparent RyuJIT replacement is not necessarily the best option. For example, it may be best to extend GC encoding to make it fit well with what LLVM can produce.
• Clang+LLVM is able to deal with SEH filters on Windows. llvm.localescape and llvm.localrecover intrinsics that were added to make the EH filters possible. It is one possible eventual strategy for implementing filters.

[yowl]

In accordance with #313 (comment), I wrote up, briefly, where LLVM(for Wasm) filter handling order was wrong in CoreRT here dotnet/corert#8283, with an idea for fixing it. I've not looked at llvm.localescape to see if that would work for all LLVM targets, or all except Wasm perhaps. Anything that makes it look more like other backends obviously better than coding workarounds.

[yowl]

I'm curious about "build a transparent RyuJIT replacement is not necessarily the best option". I'm a bit out of my depth as far as extending RyuJit is concerned to be sure, but what would be your approach @jkotas ? Can it be done in c# or is it going to require what looks like a visitor pattern implemented in cpp as above?

[yowl]

I had a read of https://github.com/dotnet/coreclr/blob/master/Documentation/botr/ryujit-tutorial.md. Looking at the "Backend phases":

Would there be an intercept somewhere along this pipeline?

[jkotas]

I think the idea would be to convert Rationalized LIR to LLVM IR. @EgorBo Is rationalized IR where your prototype is plugged in?

[yowl]

After

runtimelab/src/coreclr/jit/compiler.cpp

Line 4977 in 95de52e

rat.Run();

?

[EgorBo]

Ah, sorry for the delayed response, yeah right there. After phases where Jit removes all the assertions/nullchecks/bound checks, GT_COMMAs 🙂 However, I didn't really need LIR (e.g. I didn't use gtNext/gtPrev) and could do it before the rationalization as well. But again my prototype only handles basic control flow primitives (simple operators and basic-blocks) without GC.

[yowl]

Sounds great. Is there an opinion about where the LLVM generation should live, in c# in Ilc or in c++ land?

[EgorBo]

Sounds great. Is there an opinion about where the LLVM generation should live, in c# in Ilc or in c++ land?

LLVM C API (and hence the C# bindings) is more verbose and doesn't cover 100% of the API LLVM provides for C++ (in Mono we use mostly C API but had to switch to C++ for some things).
In my opinion it should be a standalone external C++ lib but it requires JIT to expose its classes/structs (layouts) as public APIs (a lot of work actually).

I also had a script somewhere to generate C# bindings for JIT structures/classes (at least layouts) so in theory it can be done in C# as well 🙂

[jkotas]

In my opinion it should be a standalone external C++ lib but it requires JIT to expose its classes/structs (layouts) as public APIs (a lot of work actually).

I think this should be a special build of the JIT that calls into LLVM C or C++ APIs. I do not see why we would need to expose JIT internals as public APIs for this.

[yowl]

Thinking about how to structure this in Ilc, would it make sense to try to reuse CorInfoImpl.cs, for the unmanaged callbacks for example?

[jkotas]

would it make sense to try to reuse CorInfoImpl.cs, for the unmanaged callbacks for example?

I think so

[yowl]

I made a couple of idea changes at https://github.com/yowl/runtimelab/tree/ryujit-llvm-imethodcodenode .

Idea being to introduce an IMethodCodeNode so that an LLVMMethodCodeNode can be passed to https://github.com/yowl/runtimelab/blob/08a3e45afaf941a242b0cc5161438fbba252c4ae/src/coreclr/tools/Common/JitInterface/CorInfoImpl.cs#L147. Then add a phase at https://github.com/yowl/runtimelab/blob/08a3e45afaf941a242b0cc5161438fbba252c4ae/src/coreclr/jit/compiler.cpp#L5057 to compile to LLVM which initially will just fail, so that in Ilc it can fall back to IL -> LLVM. That way the RyuJIT IR->LLVM can be built up in steps and the tests can still run - any methods that clrjit fails on, will compile with the existing IL->LLVM code. Now that #591 is merged a new clrjit_xxx_yyy_zzz.dll could be created for wasm32 (at the moment it uses clrjit_unix_x64_x64.dll which is the wrong pointer size). I don't know what affect the OS has on the JIT for the phases that this will run, so don't know if unix is the right choice. @EgorBo I did have a look around your GitHub repos in case you had checked in where you had got to but I didn't see anything. If this seems like a viable way to structure this effort, I can try to create the clrjit dll for wasm32, find a way to pass the LLVM Module into clrjit so Ilc and clrjit can add to the same module, and investigate why it currently crashes with a Run-Time Check Failure #2 - Stack around the variable 'gcPtr' was corrupted.

[yowl]

@EgorBo Did you have problems including LLVM headers into clrjit? E.g. at

runtimelab/src/coreclr/inc/holder.h

Lines 14 to 20 in fab1157

	#ifdef PAL_STDCPP_COMPAT
	#include <utility>
	#include <type_traits>
	#else
	#include "clr_std/utility"
	#include "clr_std/type_traits"
	#endif

it seems to bring in clr versions of some C++ STL templates causing duplicate definitions as LLVM brings them in from C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Professional\\VC\\Tools\\MSVC\\14.28.29333\\include\\xtr1common

[EgorBo]

@yowl Sorry for the delayed replies - the notifications were turned off for me for this repo for some reason.

Did you have problems including LLVM headers into clrjit

Yes, I couldn't solve them in a reasonable time, gave up and just did everything in a separate lib + added jit headers (stripped)
but I wish we could do it properly 🙂