Are there any alternatives to move holyjit forward on stable?

[siriux]

Hi @nbp

I've reading about holyjit and it seems that the current approach using a compiler plugin is deemed to fail.

Do you see any alternatives that could move this project forward and eventually have it working on stable? Do you have any input from the compiler team? Or should we just forget about it?

I'm not sure what I'm talking about, but maybe using macros 2.0 to get the tokens of a function, then run it through another instance of the compiler to generate MIR/LLVM IR/... and process it and embed it as data in place of the macro?

Thanks

[siriux]

I've been thinking a little more about this, and I think that to get some kind of stability it's probably better to go directly thought LLVM IR (Cretonne IR in the future) instead of MIR. This project can be really interesting in that case: https://github.com/sunfishcode/llvm2cretonne

Also, to call the second instance of the compiler you will probably need to configure it's path, and use temporary files for the input/output. It's not perfect, but it will work.

Another option would be to use the compiler programmatically, but I think this will always be unstable. Maybe, if this is successful they can commit to a simple stable api to generate LLVM IR (Cretonne IR), as it doesn't force the stabilization of anything internal (like MIR).

To run the compiler programmatically, there are some examples and information here: https://github.com/nrc/stupid-stats

What do you think?

Cheers

[nbp]

Hi @siriux,

I am currently working on rebasing it and removing the plugin on my spare time. So far I can compile HolyJit, but it no longer compile the brainfuck example due to rustc changes. I hope to get this work pushed to github in a few weeks, as soon as the brainfuck example works.

Then, HolyJit will probably remain on nightly until we have something which is worth to be used in production, and no longer experimental.

Using macros would imply that one compiles the Rust code with another version of Rustc would cause differential behaviour over time, and debugging these would be even harder that just extracting the MIR produced by Rustc.

The MIR offer some advantages over LLVM IR, such as full type information (type wrappers, alias analysis), some annotations in the code (no-op annotating blocks). On the other hand it has some disadvantages, such as not being able to reference every generated symbol. At the end, I think is better to have more higher level information than what rustc gives to LLVM.

Cretonne is designed following the same principles as an AOT compiler, even if it is optimized for memory consumption, it would certainly not be a good support for building an optimizing JIT. Thus converting LLVM IR to Cretonne sounds appealing for building an AOT, but I think JIT compilers deserve better designs than what AOT have to offers. Also Cretonne does not provide any notions of stack-maps yet, which is mandatory for doing any kind of on-stack-replacement. I might rely on Cretonne in the future, but I do not know the inner details yet.

Thanks for your interest.

[colelawrence]

Thanks for the insight, Nicholas.

…

On Thu, Mar 15, 2018, 1:34 PM Nicolas B. Pierron ***@***.***> wrote: Closed #23 <#23>. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#23 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACyjU17JMz6EXQoOZjx9G9fCtyKo4UMpks5terQwgaJpZM4Sr2To> .

[siriux]

Thanks for your detailed answer, looking forward to see your changes.

Can you comment a little bit on how do you plan to remove the plugin? How do you access the MIR code of a function?

[nbp]

As you mentioned, I run the compiler programmatically, i.e. that I am making a new program which is links against rustc internals, to inject the extra phase which was added in the plugin.

At the end, this means that you will have to replace rustc by the rustc+holyjit compiler. While I do not find this approach satisfying this is currently the only supported method for extending rustc.

I do not find this approach satisfying because I do not want to be a replacement for rustc, but only and extension for it, and renaming the binary feels too much like a replacement from my point of view.