Compilation to WASM?

[RReverser]

Miri maintainer note: this is a fun project, but not something we currently intend to support officially. To keep maintenance manageable, Miri only supports running on platforms that rustc supports running on.

Compiling the whole Rustc to WASM is a pretty big undertaking for many reasons.

However, Miri doesn't need an actual codegen and many other parts of the whole Rustc, so I wonder how realistic it would be to compile it and the pieces it depends on to WASM instead? Are there any obvious blockers?

Mostly opening this to gauge interest and estimate complexity, as I believe there is an interest in running Rust directly in the browser on playground-like websites.

P.S. Despite what I said in the first sentence, this was actually done for Clang a while ago - https://tbfleming.github.io/cib/ - which includes LLVM compiled to WASM that, in turn, generates more WASM dynamically during runtime. In theory, it should be possible to do the same for Rust, especially since they share LLVM, but for now having just an interpreter could already be an interesting starting goal.

[oli-obk]

I think a first step would be to create a new codegen backend which doesn't actually do any codegen and just dumps the metadata. That way we should be able to build a rustc which doesn't depend on llvm or other C code.

[bjorn3]

I think a first step would be to create a new codegen backend which doesn't actually do any codegen and just dumps the metadata.

I had created one in the past but it bitrotted and it was unused, so I removed it in rust-lang/rust#58847. If you copy https://github.com/bjorn3/rustc_codegen_cranelift/blob/11d816c/src/lib.rs#L182-L190 into provide and provide_extern it should work. (Also change target_features_whitelist to contain the same as cg_llvm https://github.com/rust-lang/rust/blob/2d401fb4dc89eaef5b8f31330636094f9c26b4c4/src/librustc_codegen_llvm/llvm_util.rs#L249, otherwise stdsimd wont compile.)

Another thing necessary is replacing the dlopen for loading the codegen backend with a regular extern crate. Eg replace the match at https://github.com/rust-lang/rust/blob/08bfe16129b0621bc90184f8704523d4929695ef/src/librustc_interface/util.rs#L271 with _ => || Box::new(MetadataOnlyCodegenBackend) or however you call the backend.

[bjorn3]

I want to do the same for https://github.com/bjorn3/rustc_codegen_cranelift/, but I want it to pass the rustc test suite first and cranelift doesnt support wasm output yet.

[RalfJung]

Intriguing. :) I should add one warning though: Miri isn't a fast interpreter. It's really slow. So I don't think it is actually a good environment to use to run code, I see it as more useful for debugging and testing.

But, don't let me stop you! I just felt I should give you a fair warning. And if ideas like this leak to people making Miri lightning fast while maintaining all the UB checking, I'll be even more happier. :D

[RReverser]

Miri isn't a fast interpreter. It's really slow. So I don't think it is actually a good environment to use to run code, I see it as more useful for debugging and testing.

That's understandable, but I think it's ought to be good enough for typical playground snippets :)

[RReverser]

I think a first step would be to create a new codegen backend which doesn't actually do any codegen and just dumps the metadata. That way we should be able to build a rustc which doesn't depend on llvm or other C code.

I guess that's one way, although I was wondering if Miri actually needs the main rustc crate or maybe it could be possible to depend only on some of the finer-grained rustc_* crates and avoid including codegen altogether?

[bjorn3]

The codegen backend is necessary for rustc_driver to work. Using a dummy codegen backend (<100LOC mostly copyable from the code I mentioned in #722 (comment)) is a lot easier than duplicating all the things rustc_driver does (>>1000LOC).

[RReverser]

Fair enough.

[bjorn3]

I am currently trying to compile rustc for wasm (https://github.com/bjorn3/rust/tree/compile_rustc_for_wasm), but I am hitting a compiler bug: rust-lang/rust#60540.

[RReverser]

@bjorn3 I've rebased your branch onto master, updated deps and fixed cfg's from target_env to target_os - you can check it out at https://github.com/RReverser/rust/tree/compile_rustc_for_wasm.

Eventually it compiled successfully, but then ran into the same runtime validation issue with invalid code generated by Rust.

However, I recompiled in release mode and then it passed validation!

That got me thinking it should work now, but running the generated file with wasmtime or wasmer now seems to just hang. Some infinite loop somewhere perhaps?

[RReverser]

@bjorn3 Oh... maybe it's just been taking so long (especially the compilation part). I've tried wasmer with --backend singlepass instead now, and it has actually worked!

$ ./wasmer run target/wasm32-unknown-wasi/release/rustc_binary.wasm --backend singlepass
Usage: rustc [OPTIONS] INPUT

Options:
    -h, --help          Display this message
        --cfg SPEC      Configure the compilation environment
    -L [KIND=]PATH      Add a directory to the library search path. The
                        optional KIND can be one of dependency, crate, native,
                        framework or all (the default).
    -l [KIND=]NAME      Link the generated crate(s) to the specified native
                        library NAME. The optional KIND can be one of
                        static, dylib, or framework. If omitted, dylib is
                        assumed.
        --crate-type [bin|lib|rlib|dylib|cdylib|staticlib|proc-macro]
                        Comma separated list of types of crates
                        for the compiler to emit
        --crate-name NAME
                        Specify the name of the crate being built
        --edition 2015|2018
                        Specify which edition of the compiler to use when
                        compiling code.
        --emit [asm|llvm-bc|llvm-ir|obj|metadata|link|dep-info|mir]
                        Comma separated list of types of output for the
                        compiler to emit
        --print [crate-name|file-names|sysroot|cfg|target-list|target-cpus|target-features|relocation-models|code-models|tls-models|target-spec-json|native-static-libs]
                        Comma separated list of compiler information to print
                        on stdout
    -g                  Equivalent to -C debuginfo=2
    -O                  Equivalent to -C opt-level=2
    -o FILENAME         Write output to <filename>
        --out-dir DIR   Write output to compiler-chosen filename in <dir>
        --explain OPT   Provide a detailed explanation of an error message
        --test          Build a test harness
        --target TARGET Target triple for which the code is compiled
    -W, --warn OPT      Set lint warnings
    -A, --allow OPT     Set lint allowed
    -D, --deny OPT      Set lint denied
    -F, --forbid OPT    Set lint forbidden
        --cap-lints LEVEL
                        Set the most restrictive lint level. More restrictive
                        lints are capped at this level
    -C, --codegen OPT[=VALUE]
                        Set a codegen option
    -V, --version       Print version info and exit
    -v, --verbose       Use verbose output

Additional help:
    -C help             Print codegen options
    -W help             Print 'lint' options and default settings
    -Z help             Print unstable compiler options
    --help -v           Print the full set of options rustc accepts

[bjorn3]

Oh... maybe it's just been taking so long (especially the compilation part).

Yes, it takes several minutes to compile it using wasmtime with cranelift as backend.

However, I recompiled in release mode and then it passed validation!

🎉 🎉 🎉

and it has actually worked!

I tried actually compiling something, but it errors with:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Custom { kind: Other, error: StringError("operation not supported on wasm yet") }', src/libcore/result.rs:999:5
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

I am currently trying to figure out were it errors.

[bjorn3]

Places needing patching:

• librustc_interface/util.rs: spawn_thread_pool must not spawn a thread.
• librustc/session/filesearch.rs: get_or_default_sysroot must not be called, as it needs std::env::current_exe. I am now passing an explicit --sysroot.
• librustc/session/mod.rs: build_session_ local working_dir is created using std::env::current_dir.

Edit: pushed bjorn3/rust@15f980f (based on @RReverser's branch).

Now it errors at

thread 'main' panicked at 'unknown codegen backend llvm', src/librustc_interface/util.rs:277:18

Which is expected, as I had to remove the codegen backend dynamic loader. Will try to get https://github.com/bjorn3/rustc_codegen_cranelift to work with it.

[RReverser]

Places needing patching

FWIW previously (before even filing this issue) I tried compiling rustc with Emscripten instead, which should, in theory, reduce number of these places to patch, as it supports a bit more than WASI does. Haven't gotten too far though, because I tried to build completely unpatched rustc and there were few things that still didn't compile and probably needed similar fixes as in your branch.

Which is expected, as I had to remove the codegen backend dynamic loader. Will try to get bjorn3/rustc_codegen_cranelift to work with it.

I thought the plan was to build it without any codegen, just with miri? Or do you want to build an actual full rustc?

[bjorn3]

I thought the plan was to build it without any codegen, just with miri? Or do you want to build an actual full rustc?

I want them both. :) I currently have rustc_codegen_cranelift hooked up, but rustc gives an error before rustc_codegen_cranelift can do actual codegen: can't find crate for `std` . Supporting miri will need that error to be fixed too.

[RReverser]

can't find crate for std

Yeah for that I think you'll need to do the proper build (via x.py build) to build all components. I haven't had much luck with that yet due to failures in other crates which probably also need to be patched similarly to rustc itself.

[bjorn3]

Yeah for that I think you'll need to do the proper build (via x.py build) to build all components.

Seems like it doesn't even reach the rustc version check for the libraries. I added --sysroot $(rustc --print sysroot) and disabled the rustc version check, but it still gives the same error.

[bjorn3]

Switching from wasmer to wasmtime fixed it. It even got to the beginning of codegen.

Edit: filled wasmerio/wasmer#434.

[bjorn3]

I am currently working on making miri compile for wasi, which this issue was actually about.

[bjorn3]

It seems to trap while calling the ecx.run. :(

error while processing main module ../../target/wasm32-unknown-wasi/release/rustc_binary.wasm: Instantiation error: Trap occurred while invoking start function: wasm trap at 0x2a881eb82

I pushed the wip stuff to my branch.

[RReverser]

@bjorn3 Left a comment on your MIRI commit on your branch.

[RReverser]

@bjorn3 But also, I'm not sure why rustc is now depending on miri... shouldn't it be the other way around? (like in non-WASI version)

[bjorn3]

I did that to be able to prevent having to recompile every rustc crate, which is slow and to prevent having to copy all files in the dir layout rustc wants a sysroot to be.

[RReverser]

I'm not sure I understand what you're saying... neither should be affected by which crate you compile as an entry point.

I've changed my local copy of Rust & MIRI to do just that, and got miri.wasm successfully, but yeah, also hitting some trap.

[bjorn3]

I meant that I had already compiled all crates in rust/target. When switching to miri as crate root, I would have to recompile all crates into miri/target. By keeping rustc-binary as crate root, I could reuse rust/target as target dir.

[bjorn3]

Got a new version up. This time all logic to handle using different codegen backends for the different stages, so you don't need to do a multi stage process.

• Make sure you are running on Linux.
• Clone https://github.com/bjorn3/rust
• Checkout the compile_rustc_for_wasm10 branch
• Download and extract https://github.com/WebAssembly/wasi-sdk/releases/download/wasi-sdk-20/wasi-sdk-20.0-linux.tar.gz into the rust checkout.
• Run ./x.py install to build the compiler and install it into the dist dir.
• Use wasmtime run --mapdir /::tmp --dir dist dist/bin/rustc.wasm --sysroot dist --target x86_64-unknown-linux-gnu to run rustc.

Note that linking is not supported as wasi doesn't allow spawning executables. You can use something like the following to compile using the wasm32-wasi rustc and then link using native gcc:

$ mkdir tmp
$ echo 'fn main() { println!("Hello World!"); }' | wasmtime run --mapdir /::tmp --dir dist dist/bin/rustc.wasm - --sysroot dist --target x86_64-unknown-linux-gnu -Csave-temps
$ gcc -fuse-ld=lld tmp/rmeta*/lib.rmeta tmp/rust_out.* dist/lib/rustlib/x86_64-unknown-linux-gnu/lib/lib*.rlib -o rust_out
$ ./rust_out
Hello World!

[bjorn3]

For those who don't want to build it themself here is the version I compiled. Concatenate both files together to form a .tar.xz file. I had to split and use txt as extension to workaround github limitations.

wasm_rustc.tar.xz.partaa.txt
wasm_rustc.tar.xz.partab.txt

[RealSput]

For those who don't want to build it themself here is the version I compiled. Concatenate both files together to form a .tar.xz file. I had to split and use txt as extension to workaround github limitations.

wasm_rustc.tar.xz.partaa.txt wasm_rustc.tar.xz.partab.txt

When I do that, I think it gets corrupted. I get this error trying to compile it:

Uncaught (in promise) CompileError: WebAssembly.compileStreaming(): expected magic word 00 61 73 6d, found fd 37 7a 58 @+0

This is how I concatenate it:

type rustc1.txt rustc2.txt > rustc.wasm

[bjorn3]

If you concatenate them you get an xzip compressed tarball. You need to extract this tarball to get a directory containing both rustc.wasm and the precompiled standard library that will be used by rustc. In case of browser_wasi_shim's rustc.html example you have to rename the dist directory to examples/wasm-rustc.

[beingminimal]

If you concatenate them you get an xzip compressed tarball. You need to extract this tarball to get a directory containing both rustc.wasm and the precompiled standard library that will be used by rustc. In case of browser_wasi_shim's rustc.html example you have to rename the dist directory to examples/wasm-rustc.

@bjorn3 can you please help to write blog for this and achieve this step by step for new devs also?

[bjorn3]

I don't have the time for that right now, but I will consider writing a blog post in the future. First I want to clean it up and fix a panic that happens after it already produced the object files. Also it can't produce webassembly modules right now. Only x86_64, aarch64 and s390x object files. But reporting errors and warnings does work just fine. Is that enough for you?

[jeff-hykin]

Is that enough for you?

I know its easy to forget, so from one open source dev to another, just a reminder that nothing was promised and anything you give will/should be appreciated.

I'm probably just as excited/eager as @beingminimal to eventually read a blog post from you @bjorn3 (I also don't think I can get it running just from the info in this thread). But you should do it when you have time/interest.

[beingminimal]

I don't have the time for that right now, but I will consider writing a blog post in the future. First I want to clean it up and fix a panic that happens after it already produced the object files. Also it can't produce webassembly modules right now. Only x86_64, aarch64 and s390x object files. But reporting errors and warnings does work just fine. Is that enough for you?

Hey bro, so once you complete this work we will be able to write, compile rust in browsers also and in vscode.dev also, what are your thiughts?

[bjorn3]

Compiling to working executables that can run in the browser doesn't work with this without a Webassembly backend for Cranelift (or a port of LLVM to run on wasm), but the equivalent of cargo check does work. For vscode.dev it would be necessary to write a vscode extension which uses vscode-wasm to run rustc.wasm. I have never written a vscode extension that uses vscode-wasm before so I don't know how hard that would be. I also don't know what the extension size limit is. The bundle of rustc.wasm + standard library is on the large side.

[beingminimal]

Compiling to working executables that can run in the browser doesn't work with this without a Webassembly backend for Cranelift (or a port of LLVM to run on wasm), but the equivalent of cargo check does work. For vscode.dev it would be necessary to write a vscode extension which uses vscode-wasm to run rustc.wasm. I have never written a vscode extension that uses vscode-wasm before so I don't know how hard that would be. I also don't know what the extension size limit is. The bundle of rustc.wasm + standard library is on the large side.

Hey thanks a lot for the reply.
We want to solve for people to write, compile and run rust code in vscode.dev which is the browser base.
I personally request you to study the links below. I think they have already done the required things to make this live.
We just need to connect available dots to achieve this.
https://code.visualstudio.com/blogs/2023/06/05/vscode-wasm-wasi
https://marketplace.visualstudio.com/items?itemName=ms-vscode.wasm-wasi-core
https://github.com/microsoft/vscode-wasm

I am not that much experienced otherwise i could have implemented a demo with these available things and have written blog also. But now it seems with your help it would be possible to achieve this.
A small step by step guide and github repo would be fine also.

Awaiting your reply. Thanks
@bjorn3 any thoughts bro?

[fee1-dead]

when I have time I might make a poc vscode-wasm extension

[jeff-hykin]

I have never written a vscode extension that uses vscode-wasm before so I don't know how hard that would be.

I have, its not hard cause its the same as the browser (unless VS Code has recently made changes to the API, which I doubt they would).

[bjorn3]

I did another build of rustc + cg_clif for wasm32-wasi with the latest rustc version and a couple of bugs fixed. The source code can be found at https://github.com/bjorn3/rust/tree/compile_rustc_for_wasm11 and the pre-compiled binaries at https://github.com/bjorn3/wasm-rustc (see comments.txt in the source tree for how to build it yourself) I have also updated the browser_wasi_shim example in bjorn3/browser_wasi_shim@9289c4c. It is now able to reach the exact point where the linker would be invoked again, rather than panicking during linking preparation. Compiling to wasm is tracked by bytecodealliance/wasmtime#2566.

[bjorn3]

Opened rust-lang/rust#120348 to upstream most of the build system changes, pushed rust-lang/rustc_codegen_cranelift@7d3b293 to upstream a cg_clif change and managed to somewhat reduce the diff on the compile_rustc_for_wasm13 branch. The second commit on that branch uses the wasm32-wasi-preview1-threads target to further reduce the diff, but requires wasi-threads, which browser_wasi_shim currently doesn't support.

[bjorn3]

rust-lang/rust#120348 has been merged. Opened rust-lang/rust#121392 to make patching away dlopen usage easier. I rebased the compile_rustc_for_wasm13 branch and removed a couple of unnecessary changes.

Edit: rust-lang/rust#121392 has been merged too. Rebased.

[bjorn3]

[jeff-hykin]

Got a link to the html/js for this?
This looks incredible even with all the caveats I'm sure it has!

[bjorn3]

I took https://github.com/bjorn3/browser_wasi_shim/blob/main/examples/rustc.html and made it invoke the miri I built from the second to last commit of the compile_rustc_for_wasm13 branch instead.

[jeff-hykin]

For others trying to run it, thats

• This commit: https://github.com/bjorn3/rust/tree/d8265e3fe341a6827ca97a062bcc4da6a25a4690
• And follow the instructions inside of comment.txt, but instead of the wasmtime step, take the generated dist folder and put it in the that webserver