Working asmjs and wasm targets #36339

brson · Sep 8, 2016

This patch set results in a working standard library for the asmjs-unknown-emscripten and wasm32-unknown-emscripten targets. It is based on the work of @badboy and @rschulman.

It does a few things:

Updates LLVM with the emscripten fastcomp patches, which include the pnacl IR legalizer and the asm.js backend. This patch is thought not to have any significant effect on existing targets.
Teaches rustbuild to correctly link C code with emscripten
Updates gcc-rs to work correctly with emscripten
Teaches rustbuild to run crate tests for emscripten with node
Modifies Thread::new to return an error on emscripten, to facilitate debugging a common failure mode
Modifies libtest to run in single-threaded mode for emscripten
Ignores a host of tests that don't work yet, mostly dealing with threads and I/O
Updates libc with wasm32 definitions (presently the same as asmjs)
Adds a wasm32-unknown-emscripten target that feeds the output of LLVM's asmjs backend through emcc to generate wasm

Notes and caveats:

This is only known to work with --enable-rustbuild.
The wasm32 target can't be tested correctly yet because of issues in compiletest and limitations in node emscripten-core/emscripten#4542, but hello.rs does seem to work when run on node via the binaryen interpreter
This requires an up to date installation of the emscripten sdk from its incoming branch
Unwinding is very broken
When enabling the emscripten targets jemalloc is disabled for all targets, which results in test failures for the host

Next steps are to fix the jemalloc issue, start building the two emscripten targets on the auto builders, then start producing nightlies.

#36317 tracks work on this.

Fixes #36515
Fixes #36515
Fixes #36356

rust-highfive · Sep 8, 2016

r? @alexcrichton

(rust_highfive has picked a reviewer for you, use r? to override)

brson · Sep 8, 2016

I have lingering doubts about the modification to Thread:🆕 why just this function and not others, why an error instead of panic.

brson · Sep 8, 2016

One thing to note about this patch is that debugging crate test failures is quite difficult because many failures result in aborts, not clean test failures. Often this is because of fatal errors in emscripten for various reasons; perhaps some because of the utterly broken unwinding support.

tomaka · Sep 8, 2016

In my opinion we should pass -s ERROR_ON_UNDEFINED_SYMBOLS=1 to emscripten. By default linker error generate an abort at runtime instead of compile-time. See this comment.

Other than that, there's the question of DISABLE_EXCEPTION_CATCHING. See this page.
Since Rust gives the possibility to set panic=abort, I think that this flag should be set to 0 when the panic strategy is unwind and to 1 when it is abort.

alexcrichton · Sep 8, 2016

alexcrichton reviewed Sep 8, 2016

View changes

alexcrichton · Sep 8, 2016

src/librustc_back/target/wasm32_unknown_emscripten.rs

+        .. Default::default()
+    };
+    Ok(Target {
+        llvm_target: "asmjs-unknown-emscripten".to_string(),


Should this be wasm32-unknown-emscripten?

Oh wait nevermind, this isn't using the wasm backend of LLVM.

alexcrichton · Sep 8, 2016

alexcrichton reviewed Sep 8, 2016

View changes

alexcrichton · Sep 8, 2016

src/libstd/fs.rs

@@ -1736,6 +1736,7 @@ mod tests {
    }

    #[test]
+    #[cfg_attr(target_os = "emscripten", ignore)]


Could we perhaps just #[cfg(target_os = "emscripten")] the whole module here?

alexcrichton · Sep 8, 2016

alexcrichton reviewed Sep 8, 2016

View changes

alexcrichton · Sep 8, 2016

src/libstd/net/tcp.rs

@@ -454,6 +454,7 @@ mod tests {
    }

    #[test]
+    #[cfg_attr(target_os = "emscripten", ignore)]


Perhaps like above we could ignore the entire test module here?

alexcrichton · Sep 8, 2016

alexcrichton reviewed Sep 8, 2016

View changes

alexcrichton · Sep 8, 2016

src/libstd/net/udp.rs

@@ -378,6 +378,7 @@ mod tests {
    }

    #[test]
+    #[cfg_attr(target_os = "emscripten", ignore)]


(same as above)

alexcrichton · Sep 8, 2016

alexcrichton reviewed Sep 8, 2016

View changes

alexcrichton · Sep 8, 2016

src/libstd/process.rs

@@ -819,6 +819,7 @@ mod tests {

    #[test]
    #[cfg_attr(target_os = "android", ignore)]
+    #[cfg_attr(target_os = "emscripten", ignore)]


(same as above)

alexcrichton · Sep 8, 2016

alexcrichton reviewed Sep 8, 2016

View changes

alexcrichton · Sep 8, 2016

src/libstd/sync/mpsc/mod.rs

@@ -1314,6 +1314,7 @@ mod tests {
    }

    #[test]
+    #[cfg_attr(target_os = "emscripten", ignore)]


alexcrichton · Sep 8, 2016

alexcrichton reviewed Sep 8, 2016

View changes

alexcrichton · Sep 8, 2016

src/libstd/sync/mpsc/select.rs

@@ -444,6 +444,7 @@ mod tests {
    }

    #[test]
+    #[cfg_attr(target_os = "emscripten", ignore)]


alexcrichton · Sep 8, 2016

alexcrichton reviewed Sep 8, 2016

View changes

alexcrichton · Sep 8, 2016

src/libstd/sys/unix/thread.rs

@@ -32,6 +32,10 @@ unsafe impl Sync for Thread {}
 impl Thread {
    pub unsafe fn new<'a>(stack: usize, p: Box<FnBox() + 'a>)
                          -> io::Result<Thread> {
+        if cfg!(target_os = "emscripten") {
+            return Err(io::Error::new(io::ErrorKind::Other, "emscripten does not support threads"));
+        }


Perhaps we could leave this out if it's only intended for debugging? In theory once we get panics working better the error messages would be higher quality?

alexcrichton · Sep 8, 2016

alexcrichton reviewed Sep 8, 2016

View changes

alexcrichton · Sep 8, 2016

src/libtest/lib.rs

+
+            if let Some((printio, panicio)) = oldio {
+                if let Some(printio) = printio {
+                    io::set_print(printio);


I think this may be the bug you were seeing with not printing the test name. If the override wasn't previously set it returns None, so this is never reset to the normal stdout stream.

I think you may want to change this function to just take Option<Box> and that way it can be reset back to None

alexcrichton · Sep 8, 2016

I've realized one further issue where we may not need to actually include the fastcomp backend, but I would prefer to land this ahead of that and we can decide on it later.

bors · Sep 11, 2016

☔️ The latest upstream changes (presumably #36369) made this pull request unmergeable. Please resolve the merge conflicts.

alexcrichton · Sep 12, 2016

This may also want to use the support added in #36256 to figure out which node to run rather than always using "node"

brson · Sep 14, 2016

I've got a patch to make the test changes, remove the Thread::spawn error handling, and fix libtest. Testing it now.

Still have to investigate the jemalloc problems, though this can land without fixes for that. Still have to rebase to incorporate #36256.

tomaka · Sep 14, 2016

Please don't forget my remark. You may have linking errors and not realize it.

brson · Sep 15, 2016

@tomaka Thanks for the reminder. I will definitely add the ERROR_ON_UNDEFINED_SYMBOLS=1 flag. The unwinding stuff for future work.

brson · Sep 16, 2016

@tomaka I looked into adding the flag and it looks impractical to do in this PR because every test case encounters undefined symbols related to the unwinder. I did file a bug #36515

brson · Sep 16, 2016

@alexcrichton I think this is ready to go but still running tests.

brson · Sep 30, 2016

I've confirmed this build works with the smaller LLVM patch.

brson · Sep 30, 2016

@bors r=alexcrichton

bors · Sep 30, 2016

📌 Commit afa72b5 has been approved by alexcrichton

bors · Oct 1, 2016

⌛️ Testing commit afa72b5 with merge 8b00355...

bors · Oct 1, 2016

rust-highfive assigned alexcrichton Sep 8, 2016

brson referenced this pull request Sep 8, 2016
Open
design a deployment workflow #117

alexcrichton referenced this pull request Sep 8, 2016
Closed
Does our LLVM need the fastcomp backend for emscripten? #36356

alexcrichton referenced this pull request Sep 12, 2016
Closed
WebAssembly backend for rustc #33205

joshuawarner32 referenced this pull request Sep 13, 2016
Merged
Upgrade to nightly #39

japaric referenced this pull request Sep 15, 2016
Open
Issues found so far #12

brson force-pushed the brson:emscripten-new branch 2 times, most recently from 2ea0371 to 80a1288 Sep 15, 2016

brson force-pushed the brson:emscripten-new branch from 80a1288 to 3a4ac8a Sep 16, 2016

brson force-pushed the brson:emscripten-new branch from a5f5f9d to afa72b5 Sep 30, 2016

bors merged commit afa72b5 into rust-lang:master Oct 1, 2016
2 checks passed

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details

homu Test successful
Details

This was referenced Oct 1, 2016

Merged

add Thumbs to the compiler #36874

Merged

Restrict where in the tree platform-specific cfgs may be mentioned #36807

maleadt referenced this pull request Oct 2, 2016
Closed
META: CUDA external language implementation #18338
4 of 9 tasks complete

xen0n referenced this pull request Nov 26, 2016
Open
前端问题汇总，我和蛇神解答简单问题 #100

japaric added a commit to rust-lang-nursery/compiler-builtins that referenced this pull request Feb 6, 2017

Don't build any native compiler-builtin components for emscripten
rust-lang/rust#36339

c30dcb0

rust-lang/rust

Join GitHub today

Working asmjs and wasm targets #36339

Conversation

brson commented Sep 8, 2016 • edited

rust-highfive assigned alexcrichton Sep 8, 2016

This comment has been minimized.

rust-highfive commented Sep 8, 2016

This comment has been minimized.

brson commented Sep 8, 2016

This comment has been minimized.

brson commented Sep 8, 2016

This comment has been minimized.

tomaka commented Sep 8, 2016

brson referenced this pull request Sep 8, 2016

alexcrichton reviewed Sep 8, 2016 View changes

This comment has been minimized.

This comment has been minimized.

alexcrichton reviewed Sep 8, 2016 View changes

This comment has been minimized.

alexcrichton reviewed Sep 8, 2016 View changes

This comment has been minimized.

alexcrichton reviewed Sep 8, 2016 View changes

This comment has been minimized.

alexcrichton reviewed Sep 8, 2016 View changes

This comment has been minimized.

alexcrichton reviewed Sep 8, 2016 View changes

This comment has been minimized.

alexcrichton reviewed Sep 8, 2016 View changes

This comment has been minimized.

alexcrichton reviewed Sep 8, 2016 View changes

This comment has been minimized.

alexcrichton reviewed Sep 8, 2016 View changes

This comment has been minimized.

alexcrichton referenced this pull request Sep 8, 2016

This comment has been minimized.

alexcrichton commented Sep 8, 2016

This comment has been minimized.

bors commented Sep 11, 2016

alexcrichton referenced this pull request Sep 12, 2016

This comment has been minimized.

alexcrichton commented Sep 12, 2016

joshuawarner32 referenced this pull request Sep 13, 2016

This comment has been minimized.

brson commented Sep 14, 2016

This comment has been minimized.

tomaka commented Sep 14, 2016

japaric referenced this pull request Sep 15, 2016

This comment has been minimized.

brson commented Sep 15, 2016

brson force-pushed the brson:emscripten-new branch 2 times, most recently from 2ea0371 to 80a1288 Sep 15, 2016

This comment has been minimized.

brson commented Sep 16, 2016

brson force-pushed the brson:emscripten-new branch from 80a1288 to 3a4ac8a Sep 16, 2016

This comment has been minimized.

brson commented Sep 16, 2016

brson added some commits Sep 7, 2016

brson force-pushed the brson:emscripten-new branch from a5f5f9d to afa72b5 Sep 30, 2016

This comment has been minimized.

brson commented Sep 30, 2016

This comment has been minimized.

brson commented Sep 30, 2016

This comment has been minimized.

bors commented Sep 30, 2016

This comment has been minimized.

bors commented Oct 1, 2016

bors added a commit that referenced this pull request Oct 1, 2016

This comment has been minimized.

bors commented Oct 1, 2016

Hide details View details bors merged commit afa72b5 into rust-lang:master Oct 1, 2016 2 checks passed

2 checks passed

This was referenced Oct 1, 2016

maleadt referenced this pull request Oct 2, 2016

xen0n referenced this pull request Nov 26, 2016

japaric added a commit to rust-lang-nursery/compiler-builtins that referenced this pull request Feb 6, 2017

brson commented Sep 8, 2016 •

edited

alexcrichton reviewed Sep 8, 2016

View changes

alexcrichton reviewed Sep 8, 2016

View changes

alexcrichton reviewed Sep 8, 2016

View changes

alexcrichton reviewed Sep 8, 2016

View changes

alexcrichton reviewed Sep 8, 2016

View changes

alexcrichton reviewed Sep 8, 2016

View changes

alexcrichton reviewed Sep 8, 2016

View changes

alexcrichton reviewed Sep 8, 2016

View changes

alexcrichton reviewed Sep 8, 2016

View changes

brson force-pushed the brson:emscripten-new branch 2 times, most recently from `2ea0371` to `80a1288` Sep 15, 2016

brson force-pushed the brson:emscripten-new branch from `80a1288` to `3a4ac8a` Sep 16, 2016

brson force-pushed the brson:emscripten-new branch from `a5f5f9d` to `afa72b5` Sep 30, 2016

bors merged commit `afa72b5` into rust-lang:master Oct 1, 2016
2 checks passed