Migrate wasm target to LLVM wasm backend #38804

Closed
brson opened this Issue Jan 3, 2017 · 12 comments

Comments

Projects
None yet
9 participants
@brson
Contributor

brson commented Jan 3, 2017

At some point we must move rustc off the LLVM fastcomp backend. Maintaining it in sync with emcc is exceedingly burdensome. The only viable solution I see any time soon is to use the nascent LLVM wasm backend, which is not yet generally usable. I expect that we will switch to that backend, emitting wasm into rlibs instead of LLLVM IR, then link multiple wasm modules together via emcc. We may need to hack on emcc a bit to teach it to link wasm modules directly, not sure.

Even though the LLVM wasm backend is not fully working yet, somebody could go ahead and start doing the conversion and seeing what the results are like.

cc LLVM upgrade #37609

@brson brson referenced this issue Jan 3, 2017

Closed

Tracking issue for wasm support #38805

11 of 15 tasks complete
@rkruppe

This comment has been minimized.

Show comment
Hide comment
@rkruppe

rkruppe Jan 3, 2017

Contributor

re: storing wasm in rlibs and hacking emcc to make it link those, this would neatly be solved by MIR-only rlibs. That makes wasm the second backend that could use this (cc #38787) so I'm starting to think we should have an issue or something to collect these use cases. cc @michaelwoerister

Contributor

rkruppe commented Jan 3, 2017

re: storing wasm in rlibs and hacking emcc to make it link those, this would neatly be solved by MIR-only rlibs. That makes wasm the second backend that could use this (cc #38787) so I'm starting to think we should have an issue or something to collect these use cases. cc @michaelwoerister

@Eh2406

This comment has been minimized.

Show comment
Hide comment
@Eh2406

Eh2406 Apr 26, 2017

Contributor

What Is this up to? How does the update to LLVM 4.0 change things?

Contributor

Eh2406 commented Apr 26, 2017

What Is this up to? How does the update to LLVM 4.0 change things?

@brson

This comment has been minimized.

Show comment
Hide comment
@brson

brson Jun 2, 2017

Contributor

@Eh2406 Still no progress. The LLVM 4.0 upgrade only inches us closer, but it still doesn't have a production ready backend. Since the backend does exist (I think) in the LLVM we're on, we can start enabling it for testing purposes.

Contributor

brson commented Jun 2, 2017

@Eh2406 Still no progress. The LLVM 4.0 upgrade only inches us closer, but it still doesn't have a production ready backend. Since the backend does exist (I think) in the LLVM we're on, we can start enabling it for testing purposes.

@tlively

This comment has been minimized.

Show comment
Hide comment
@tlively

tlively Jun 5, 2017

I am currently working on using the LLVM backend behind a new target triple. The current wasm target is wasm32-unknown-emscripten, so for the new one I thought I would use wasm32-unknown-unknown. This will let us start using the LLVM backend in testing, but I was wondering what the plan is beyond that. Are we trying to remove the emscripten wasm target completely? If so, do we want to keep the asm.js target? Getting rid of both emscripten targets removes our dependency on FastComp, but if someone needs the asm.js target then this clearly isn't a possibility.

tlively commented Jun 5, 2017

I am currently working on using the LLVM backend behind a new target triple. The current wasm target is wasm32-unknown-emscripten, so for the new one I thought I would use wasm32-unknown-unknown. This will let us start using the LLVM backend in testing, but I was wondering what the plan is beyond that. Are we trying to remove the emscripten wasm target completely? If so, do we want to keep the asm.js target? Getting rid of both emscripten targets removes our dependency on FastComp, but if someone needs the asm.js target then this clearly isn't a possibility.

@brson

This comment has been minimized.

Show comment
Hide comment
@brson

brson Jun 12, 2017

Contributor

@tlively I'd prefer to both retain the asm.js target and drop fastcomp, which I know is pretty hard; the reason being that I fear that production users will want a JS fallback for browsers missing WASM support. To achieve this ultimately I'd expect to drop fastcomp and employ a wasm->asm.js translator.

That said, wasm support is rolling out fast, so this concern may end up being unfounded.

I'd suggest the path forward should look roughly like

  • Get wasm32-unknown-unknown up to feature / performance parity with wasm32-unknown-emscripten
  • Replace the fastcomp backend in wasm32-unknown-emscripten with the wasm backend, leaving wasm32-unknown-asmjs in place
  • Punt on the question of what to do with the asmjs target until the story becomes clearer
Contributor

brson commented Jun 12, 2017

@tlively I'd prefer to both retain the asm.js target and drop fastcomp, which I know is pretty hard; the reason being that I fear that production users will want a JS fallback for browsers missing WASM support. To achieve this ultimately I'd expect to drop fastcomp and employ a wasm->asm.js translator.

That said, wasm support is rolling out fast, so this concern may end up being unfounded.

I'd suggest the path forward should look roughly like

  • Get wasm32-unknown-unknown up to feature / performance parity with wasm32-unknown-emscripten
  • Replace the fastcomp backend in wasm32-unknown-emscripten with the wasm backend, leaving wasm32-unknown-asmjs in place
  • Punt on the question of what to do with the asmjs target until the story becomes clearer
@tlively

This comment has been minimized.

Show comment
Hide comment
@tlively

tlively Jun 13, 2017

This plan sounds reasonable to me. I don't know of work on a wasm -> asm.js translator, but that sounds like a tool that could be reasonably expected to exist as wasm picks up steam.

tlively commented Jun 13, 2017

This plan sounds reasonable to me. I don't know of work on a wasm -> asm.js translator, but that sounds like a tool that could be reasonably expected to exist as wasm picks up steam.

@LukasKalbertodt

This comment has been minimized.

Show comment
Hide comment
@LukasKalbertodt

LukasKalbertodt Jun 13, 2017

Contributor

There is some work on a wasm -> asm.js translator, named wasm2asm in the official binaryen repo. But apparently it's not that straightforward to do, as explained in the FAQ (second question).

There is also the possibility to write a wasm-interpreter in JavaScript (probably in asm.js). This seems to be supported by binaryen (click & click (search for 'interpret')), but I'm not really sure.

Get wasm32-unknown-unknown up to feature / performance parity with wasm32-unknown-emscripten

Who is working on that? How could one help? Is there an IRC channel just about the Rust-wasm story?

Contributor

LukasKalbertodt commented Jun 13, 2017

There is some work on a wasm -> asm.js translator, named wasm2asm in the official binaryen repo. But apparently it's not that straightforward to do, as explained in the FAQ (second question).

There is also the possibility to write a wasm-interpreter in JavaScript (probably in asm.js). This seems to be supported by binaryen (click & click (search for 'interpret')), but I'm not really sure.

Get wasm32-unknown-unknown up to feature / performance parity with wasm32-unknown-emscripten

Who is working on that? How could one help? Is there an IRC channel just about the Rust-wasm story?

@tlively

This comment has been minimized.

Show comment
Hide comment
@tlively

tlively Jun 13, 2017

Who is working on that? How could one help? Is there an IRC channel just about the Rust-wasm story?

I am! I'm not sure how the work could be parallelized, though. There's no IRC channel or other dedicated communication that I know of, but a PR with some related discussion is #42571.

tlively commented Jun 13, 2017

Who is working on that? How could one help? Is there an IRC channel just about the Rust-wasm story?

I am! I'm not sure how the work could be parallelized, though. There's no IRC channel or other dedicated communication that I know of, but a PR with some related discussion is #42571.

@eholk

This comment has been minimized.

Show comment
Hide comment
@eholk

eholk Jun 13, 2017

Contributor

Maybe it's time to start the #rust-wasm IRC channel?

Contributor

eholk commented Jun 13, 2017

Maybe it's time to start the #rust-wasm IRC channel?

bors added a commit that referenced this issue Jun 20, 2017

Auto merge of #42571 - tlively:wasm-dev, r=alexcrichton
Enable wasm LLVM backend

Enables compilation to WebAssembly with the LLVM backend using the target triple "wasm32-unknown-unknown". This is the beginning of my work on #38804.

**edit:** The new new target is now wasm32-experimental-emscripten instead of wasm32-unknown-unknown.
@WillemMali

This comment has been minimized.

Show comment
Hide comment
@WillemMali

WillemMali Jul 15, 2017

Are there wasm32-experimental-emscripten binaries available in rustup right now, and if not what needs to be done to get to that point?

I'd like to try putting together something that generates HTML serverside and also generates clientside wasm to serve along with that, from a single source file. My idea was to use a combination of Cargo features, a build.rs that invokes Cargo with the wasm32 target and include_bytes.

I'm not a cargo hacker so it's a pain to figure out how to use this patchset from source. I tried installing it with rustup but it looks to be unavailable:

error: toolchain 'nightly-x86_64-unknown-linux-gnu' does not contain component 'rust-std' for target 'wasm32-experimental-emscripten'

Any thoughts / instructions would be appreciated.

WillemMali commented Jul 15, 2017

Are there wasm32-experimental-emscripten binaries available in rustup right now, and if not what needs to be done to get to that point?

I'd like to try putting together something that generates HTML serverside and also generates clientside wasm to serve along with that, from a single source file. My idea was to use a combination of Cargo features, a build.rs that invokes Cargo with the wasm32 target and include_bytes.

I'm not a cargo hacker so it's a pain to figure out how to use this patchset from source. I tried installing it with rustup but it looks to be unavailable:

error: toolchain 'nightly-x86_64-unknown-linux-gnu' does not contain component 'rust-std' for target 'wasm32-experimental-emscripten'

Any thoughts / instructions would be appreciated.

@tlively

This comment has been minimized.

Show comment
Hide comment
@tlively

tlively Jul 17, 2017

@WillemMali if you want to compile to wasm right now, the best way to do that is with the wasm32-unknown-emscripten target. this is a good guide on how to do that.

wasm32-experimental-emscripten is truly experimental right now. Among other things, using it currently requires you to build rustc yourself locally, and there is no rustup support for it. However, I have gotten a few questions about using it, so I will write up some documentation and link to it here.

tlively commented Jul 17, 2017

@WillemMali if you want to compile to wasm right now, the best way to do that is with the wasm32-unknown-emscripten target. this is a good guide on how to do that.

wasm32-experimental-emscripten is truly experimental right now. Among other things, using it currently requires you to build rustc yourself locally, and there is no rustup support for it. However, I have gotten a few questions about using it, so I will write up some documentation and link to it here.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Aug 27, 2017

Member

I'm going to close this in favor of #44006

Member

alexcrichton commented Aug 27, 2017

I'm going to close this in favor of #44006

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment