New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bootstrapping `rustc` with `wee_alloc` as the global allocator should work #10

Open
fitzgen opened this Issue Feb 2, 2018 · 11 comments

Comments

Projects
None yet
2 participants
@fitzgen
Member

fitzgen commented Feb 2, 2018

This would be a nice smoke test :)

@fitzgen fitzgen added the help wanted label Feb 2, 2018

@pepyakin

This comment has been minimized.

Collaborator

pepyakin commented Feb 6, 2018

Running ./x.py test --keep-stage 0 --target wasm32-unknown-unknown uncover following test fails

failures:

---- [ui] ui/deprecated-macro_escape-inner.rs stdout ----

error: test run failed!
status: exit code: 101
command: "/usr/local/bin/node" "/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js" "/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/test/ui/deprecated-macro_escape-inner.stage2-wasm32-unknown-unknown.wasm"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
RuntimeError: unreachable
    at <WASM UNNAMED> (<WASM>[50]+271)
    at <WASM UNNAMED> (<WASM>[189]+76)
    at Object.<anonymous> (/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js:130:20)
    at Module._compile (module.js:573:30)
    at Object.Module._extensions..js (module.js:584:10)
    at Module.load (module.js:507:32)
    at tryModuleLoad (module.js:470:12)
    at Function.Module._load (module.js:462:3)
    at Function.Module.runMain (module.js:609:10)
    at startup (bootstrap_node.js:158:16)
    at bootstrap_node.js:598:3

------------------------------------------

thread '[ui] ui/deprecated-macro_escape-inner.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2883:9
note: Run with `RUST_BACKTRACE=1` for a backtrace.

---- [ui] ui/deprecated-macro_escape.rs stdout ----

error: test run failed!
status: exit code: 101
command: "/usr/local/bin/node" "/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js" "/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/test/ui/deprecated-macro_escape.stage2-wasm32-unknown-unknown.wasm"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
RuntimeError: unreachable
    at <WASM UNNAMED> (<WASM>[50]+271)
    at <WASM UNNAMED> (<WASM>[189]+76)
    at Object.<anonymous> (/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js:130:20)
    at Module._compile (module.js:573:30)
    at Object.Module._extensions..js (module.js:584:10)
    at Module.load (module.js:507:32)
    at tryModuleLoad (module.js:470:12)
    at Function.Module._load (module.js:462:3)
    at Function.Module.runMain (module.js:609:10)
    at startup (bootstrap_node.js:158:16)
    at bootstrap_node.js:598:3

------------------------------------------

thread '[ui] ui/deprecated-macro_escape.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2883:9

---- [ui] ui/deriving-meta-empty-trait-list.rs stdout ----

error: test run failed!
status: exit code: 101
command: "/usr/local/bin/node" "/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js" "/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/test/ui/deriving-meta-empty-trait-list.stage2-wasm32-unknown-unknown.wasm"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
RuntimeError: unreachable
    at <WASM UNNAMED> (<WASM>[50]+271)
    at <WASM UNNAMED> (<WASM>[189]+76)
    at Object.<anonymous> (/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js:130:20)
    at Module._compile (module.js:573:30)
    at Object.Module._extensions..js (module.js:584:10)
    at Module.load (module.js:507:32)
    at tryModuleLoad (module.js:470:12)
    at Function.Module._load (module.js:462:3)
    at Function.Module.runMain (module.js:609:10)
    at startup (bootstrap_node.js:158:16)
    at bootstrap_node.js:598:3

------------------------------------------

thread '[ui] ui/deriving-meta-empty-trait-list.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2883:9

---- [ui] ui/enum-size-variance.rs stdout ----

error: test run failed!
status: exit code: 101
command: "/usr/local/bin/node" "/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js" "/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/test/ui/enum-size-variance.stage2-wasm32-unknown-unknown.wasm"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
RuntimeError: unreachable
    at <WASM UNNAMED> (<WASM>[50]+271)
    at <WASM UNNAMED> (<WASM>[189]+76)
    at Object.<anonymous> (/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js:130:20)
    at Module._compile (module.js:573:30)
    at Object.Module._extensions..js (module.js:584:10)
    at Module.load (module.js:507:32)
    at tryModuleLoad (module.js:470:12)
    at Function.Module._load (module.js:462:3)
    at Function.Module.runMain (module.js:609:10)
    at startup (bootstrap_node.js:158:16)
    at bootstrap_node.js:598:3

------------------------------------------

thread '[ui] ui/enum-size-variance.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2883:9

---- [ui] ui/issue-19100.rs stdout ----

error: test run failed!
status: exit code: 101
command: "/usr/local/bin/node" "/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js" "/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/test/ui/issue-19100.stage2-wasm32-unknown-unknown.wasm"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
RuntimeError: unreachable
    at <WASM UNNAMED> (<WASM>[50]+271)
    at <WASM UNNAMED> (<WASM>[189]+76)
    at Object.<anonymous> (/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js:130:20)
    at Module._compile (module.js:573:30)
    at Object.Module._extensions..js (module.js:584:10)
    at Module.load (module.js:507:32)
    at tryModuleLoad (module.js:470:12)
    at Function.Module._load (module.js:462:3)
    at Function.Module.runMain (module.js:609:10)
    at startup (bootstrap_node.js:158:16)
    at bootstrap_node.js:598:3

------------------------------------------

thread '[ui] ui/issue-19100.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2883:9

---- [ui] ui/issue-43806.rs stdout ----

error: test run failed!
status: exit code: 101
command: "/usr/local/bin/node" "/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js" "/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/test/ui/issue-43806.stage2-wasm32-unknown-unknown.wasm"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
RuntimeError: unreachable
    at <WASM UNNAMED> (<WASM>[50]+271)
    at <WASM UNNAMED> (<WASM>[189]+76)
    at Object.<anonymous> (/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js:130:20)
    at Module._compile (module.js:573:30)
    at Object.Module._extensions..js (module.js:584:10)
    at Module.load (module.js:507:32)
    at tryModuleLoad (module.js:470:12)
    at Function.Module._load (module.js:462:3)
    at Function.Module.runMain (module.js:609:10)
    at startup (bootstrap_node.js:158:16)
    at bootstrap_node.js:598:3

------------------------------------------

thread '[ui] ui/issue-43806.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2883:9

---- [ui] ui/nll/borrow-use-issue-46875.rs stdout ----

error: test run failed!
status: exit code: 101
command: "/usr/local/bin/node" "/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js" "/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/test/ui/nll/borrow-use-issue-46875.stage2-wasm32-unknown-unknown.wasm"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
RuntimeError: unreachable
    at <WASM UNNAMED> (<WASM>[49]+271)
    at <WASM UNNAMED> (<WASM>[1]+145)
    at Object.<anonymous> (/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js:130:20)
    at Module._compile (module.js:573:30)
    at Object.Module._extensions..js (module.js:584:10)
    at Module.load (module.js:507:32)
    at tryModuleLoad (module.js:470:12)
    at Function.Module._load (module.js:462:3)
    at Function.Module.runMain (module.js:609:10)
    at startup (bootstrap_node.js:158:16)
    at bootstrap_node.js:598:3

------------------------------------------

thread '[ui] ui/nll/borrow-use-issue-46875.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2883:9

---- [ui] ui/path-lookahead.rs stdout ----

error: test run failed!
status: exit code: 101
command: "/usr/local/bin/node" "/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js" "/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/test/ui/path-lookahead.stage2-wasm32-unknown-unknown.wasm"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
RuntimeError: unreachable
    at <WASM UNNAMED> (<WASM>[50]+271)
    at <WASM UNNAMED> (<WASM>[189]+76)
    at Object.<anonymous> (/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js:130:20)
    at Module._compile (module.js:573:30)
    at Object.Module._extensions..js (module.js:584:10)
    at Module.load (module.js:507:32)
    at tryModuleLoad (module.js:470:12)
    at Function.Module._load (module.js:462:3)
    at Function.Module.runMain (module.js:609:10)
    at startup (bootstrap_node.js:158:16)
    at bootstrap_node.js:598:3

------------------------------------------

thread '[ui] ui/path-lookahead.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2883:9

---- [ui] ui/span/unused-warning-point-at-signature.rs stdout ----

error: test run failed!
status: exit code: 101
command: "/usr/local/bin/node" "/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js" "/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/test/ui/span/unused-warning-point-at-signature.stage2-wasm32-unknown-unknown.wasm"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
RuntimeError: unreachable
    at <WASM UNNAMED> (<WASM>[50]+271)
    at <WASM UNNAMED> (<WASM>[189]+76)
    at Object.<anonymous> (/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js:130:20)
    at Module._compile (module.js:573:30)
    at Object.Module._extensions..js (module.js:584:10)
    at Module.load (module.js:507:32)
    at tryModuleLoad (module.js:470:12)
    at Function.Module._load (module.js:462:3)
    at Function.Module.runMain (module.js:609:10)
    at startup (bootstrap_node.js:158:16)
    at bootstrap_node.js:598:3

------------------------------------------

thread '[ui] ui/span/unused-warning-point-at-signature.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2883:9

---- [ui] ui/test-should-panic-attr.rs stdout ----

error: test run failed!
status: exit code: 101
command: "/usr/local/bin/node" "/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js" "/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/test/ui/test-should-panic-attr.stage2-wasm32-unknown-unknown.wasm"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
RuntimeError: unreachable
    at <WASM UNNAMED> (<WASM>[257]+280)
    at <WASM UNNAMED> (<WASM>[431]+76)
    at Object.<anonymous> (/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js:130:20)
    at Module._compile (module.js:573:30)
    at Object.Module._extensions..js (module.js:584:10)
    at Module.load (module.js:507:32)
    at tryModuleLoad (module.js:470:12)
    at Function.Module._load (module.js:462:3)
    at Function.Module.runMain (module.js:609:10)
    at startup (bootstrap_node.js:158:16)
    at bootstrap_node.js:598:3

------------------------------------------

thread '[ui] ui/test-should-panic-attr.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2883:9


failures:
    [ui] ui/deprecated-macro_escape-inner.rs
    [ui] ui/deprecated-macro_escape.rs
    [ui] ui/deriving-meta-empty-trait-list.rs
    [ui] ui/enum-size-variance.rs
    [ui] ui/issue-19100.rs
    [ui] ui/issue-43806.rs
    [ui] ui/nll/borrow-use-issue-46875.rs
    [ui] ui/path-lookahead.rs
    [ui] ui/span/unused-warning-point-at-signature.rs
    [ui] ui/test-should-panic-attr.rs

test result: FAILED. 966 passed; 10 failed; 6 ignored; 0 measured; 0 filtered out

thread 'main' panicked at 'Some tests failed', tools/compiletest/src/main.rs:476:22

However, I'm exteremely not sure that I set up things well.

You can take a look at the change here

@fitzgen

This comment has been minimized.

Member

fitzgen commented Feb 6, 2018

Did you build with debug info? I'd expect that to give function names rather than <WASM>[123]+456.

I was imagining trying to bootstrap rustc with the unix version of wee_alloc -- I'm unsure if rustc will bootstrap with dlmalloc, or any other allocator, when targeting wasm.

@pepyakin

This comment has been minimized.

Collaborator

pepyakin commented Feb 6, 2018

I was imagining trying to bootstrap rustc with the unix version of wee_alloc

Ahaha, I should have figure it out! Will try : )

Did you build with debug info? I'd expect that to give function names rather than [123]+456.

Nope, im even not sure how to do it. But yeah, AFAIK, debug info + latest node should show mangled names on traps.

@pepyakin

This comment has been minimized.

Collaborator

pepyakin commented Feb 7, 2018

I'm not sure what this means, but compiling stage1 with stage0 which uses wee_alloc is running for too long (2hrs) on Compiling rustc_tsan v0.0.0 (.../rust/src/librustc_tsan). top reports that 1 core is utilized by 100%.

Can it be just so awfuly slow?

UPD:

CPU utilization is not constant 100% and memory consumption seems to be raising (although, very slowly, from 250 mb to 390 in 1hr)

@fitzgen

This comment has been minimized.

Member

fitzgen commented Feb 7, 2018

I've never actually bootstrapped rustc myself, so I don't know what is a reasonably amount of time...

Maybe try a normal bootstrap and compare?

@fitzgen

This comment has been minimized.

Member

fitzgen commented Feb 7, 2018

Just asked aturon, and he said he expects something closer to half an hour...

Seems like we have some low hanging perf work to do!

@pepyakin

This comment has been minimized.

Collaborator

pepyakin commented Feb 7, 2018

Yeah usualy it takea around 0.5 to get stage2 with default settings. In this case it takes more that two hours to compile single crate!

@pepyakin

This comment has been minimized.

Collaborator

pepyakin commented Feb 7, 2018

(it still didnt complete yet)

@fitzgen

This comment has been minimized.

Member

fitzgen commented Feb 7, 2018

(it still didnt complete yet)

lol... this is what happens when there's been zero performance work yet...

@pepyakin

This comment has been minimized.

Collaborator

pepyakin commented Feb 7, 2018

Aaaaaaand it's done! Sort of

error: Could not compile `core`.

Caused by:
  process didn't exit successfully: `/Volumes/pepa-cloud/dev/rust/build/bootstrap/debug/rustc --crate-name core libcore/lib.rs --error-format json --crate-type lib --emit=dep-info,link -C opt-level=2 -C metadata=807363e85b1c090e -C extra-filename=-807363e85b1c090e --out-dir /Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/stage1-std/x86_64-apple-darwin/release/deps --target x86_64-apple-darwin -L dependency=/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/stage1-std/x86_64-apple-darwin/release/deps -L dependency=/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/stage1-std/release/deps` (signal: 4, SIGILL: illegal instruction)
thread 'main' panicked at 'command did not execute successfully: "/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/stage0/bin/cargo" "build" "--target" "x86_64-apple-darwin" "-j" "8" "--release" "--features" "panic-unwind jemalloc backtrace" "--manifest-path" "/Volumes/pepa-cloud/dev/rust/src/libstd/Cargo.toml" "--message-format" "json"
expected success, got: exit code: 101', bootstrap/compile.rs:1061:9
note: Run with `RUST_BACKTRACE=1` for a backtrace.
failed to run: /Volumes/pepa-cloud/dev/rust/build/bootstrap/debug/bootstrap build
Build completed unsuccessfully in 4:53:19

(and yes, it turns out it stucked on core crate and not on rustc_tsan as I initially thought)

@fitzgen

This comment has been minimized.

Member

fitzgen commented Feb 7, 2018

Looks like we have some work cut out for us :)

@fitzgen fitzgen changed the title from Try bootstrapping `rustc` with `wee_alloc` as the global allocator to Bootstrapping `rustc` with `wee_alloc` as the global allocator should work Feb 7, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment