Default compiler backend

[arctic-hen7]

It's well-known among Perseus users that our compile times aren't great, which is mainly due to the Rust compiler being fairly slow. However, there is a project that can help! The linked repo provides a lightning-fast compiler backend based on Cranelift, rather than LLVM, which has, in my testing, cut compile times in development by 75%.

I'm currently (#160) making significant improvements to the CLI, one of which is installing external tool dependencies (e.g. wasm-opt) automatically behind the scenes (local to the project though), and I want to download that Cranelift backend too, and then provide the user with the option to use it in development. Note that this backend should never be used in production, since its main purpose is to minimise compile-times (we'll stick with the default compiler backend for release builds). The question now is about the default behavior.

[erlend-sh]

Maybe @bjorn3 has some input on this?

[bjorn3]

For code running in the browser using wasm, cg_clif is not usable as Cranelift doesn't have a WASM backend (bytecodealliance/wasmtime#2566 is the tracking issue for adding one). For code running on the server it should work. However there are still a couple of that make it less suitable as default rather than opt-in option. First of cg_clif requires nightly rustc and isn't shipped with rustc itself yet (cc rust-lang/rust#81746). Windows support is also rather poor (only the MinGW toolchain works at the moment.). And finally using cg_clif may be a slowdown in some cases as cg_clif doesn't yet support parallel compilation (https://github.com/bjorn3/rustc_codegen_cranelift/issues/652).

[arctic-hen7]

Thank you for the input, much appreciated! I'm aware of the Wasm limitations, but the performance gains on the engine-side are massive compared to stable, or even nightly. (A lot of changes in Perseus only involve recompiling the engine anyway, since most of the code is extensively target-gated.) It would be great if it shipped with rustc, but I'm happy to use it before then I think. What's involved in getting it to work on Windows exactly?

With regards to the nightly requirement, if this were enabled by default, the Cranelift backend would be downloaded, and then only used if the user is using nightly. (Which they would be recommended to.)

By the way, thank you for building a really awesome tool! You have given me back possibly years of my life in compile-times!

[bjorn3]

What's involved in getting it to work on Windows exactly?

Mostly implementing support for PE thread local storage in Cranelift I think. I believe there are also some abi differences in compiler-builtins for windows.

With regards to the nightly requirement, if this were enabled by default, the Cranelift backend would be downloaded, and then only used if the user is using nightly. (Which they would be recommended to.)

The backend needs to match the rustc version exactly as it is linked to rustc internal crates. Doing something like this would likely require shipping a bundle of rustc and cg_clif. There are already bundles of cg_clif without rustc created by CI, but CI artifacts unfortunately can only be downloaded when logged in on github.

By the way, thank you for building a really awesome tool! You have given me back possibly years of my life in compile-times!

💖

[arctic-hen7]

I see. Alright, probably best to make this opt-in then! For the artifacts, I was considering making a mirror repo that releases based on your GH actions... If you could add releases though, maybe based on compiler version, that would be extremely helpful! (Happy to help with this if you'd like.)

From there, it shouldn't be too hard to download the right release based on the user's rustc version and prompt them if it's too old or they're not using nightly (if it's opt-in).