Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bootstrapping rustc with wee_alloc as the global allocator should work #10

Open
fitzgen opened this issue Feb 2, 2018 · 11 comments
Open

Bootstrapping rustc with wee_alloc as the global allocator should work #10

fitzgen opened this issue Feb 2, 2018 · 11 comments

Comments

@fitzgen
Copy link
Member

@fitzgen fitzgen commented Feb 2, 2018

This would be a nice smoke test :)

@pepyakin
Copy link
Member

@pepyakin pepyakin commented Feb 6, 2018

Running ./x.py test --keep-stage 0 --target wasm32-unknown-unknown uncover following test fails

failures:

---- [ui] ui/deprecated-macro_escape-inner.rs stdout ----

error: test run failed!
status: exit code: 101
command: "/usr/local/bin/node" "/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js" "/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/test/ui/deprecated-macro_escape-inner.stage2-wasm32-unknown-unknown.wasm"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
RuntimeError: unreachable
    at <WASM UNNAMED> (<WASM>[50]+271)
    at <WASM UNNAMED> (<WASM>[189]+76)
    at Object.<anonymous> (/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js:130:20)
    at Module._compile (module.js:573:30)
    at Object.Module._extensions..js (module.js:584:10)
    at Module.load (module.js:507:32)
    at tryModuleLoad (module.js:470:12)
    at Function.Module._load (module.js:462:3)
    at Function.Module.runMain (module.js:609:10)
    at startup (bootstrap_node.js:158:16)
    at bootstrap_node.js:598:3

------------------------------------------

thread '[ui] ui/deprecated-macro_escape-inner.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2883:9
note: Run with `RUST_BACKTRACE=1` for a backtrace.

---- [ui] ui/deprecated-macro_escape.rs stdout ----

error: test run failed!
status: exit code: 101
command: "/usr/local/bin/node" "/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js" "/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/test/ui/deprecated-macro_escape.stage2-wasm32-unknown-unknown.wasm"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
RuntimeError: unreachable
    at <WASM UNNAMED> (<WASM>[50]+271)
    at <WASM UNNAMED> (<WASM>[189]+76)
    at Object.<anonymous> (/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js:130:20)
    at Module._compile (module.js:573:30)
    at Object.Module._extensions..js (module.js:584:10)
    at Module.load (module.js:507:32)
    at tryModuleLoad (module.js:470:12)
    at Function.Module._load (module.js:462:3)
    at Function.Module.runMain (module.js:609:10)
    at startup (bootstrap_node.js:158:16)
    at bootstrap_node.js:598:3

------------------------------------------

thread '[ui] ui/deprecated-macro_escape.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2883:9

---- [ui] ui/deriving-meta-empty-trait-list.rs stdout ----

error: test run failed!
status: exit code: 101
command: "/usr/local/bin/node" "/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js" "/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/test/ui/deriving-meta-empty-trait-list.stage2-wasm32-unknown-unknown.wasm"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
RuntimeError: unreachable
    at <WASM UNNAMED> (<WASM>[50]+271)
    at <WASM UNNAMED> (<WASM>[189]+76)
    at Object.<anonymous> (/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js:130:20)
    at Module._compile (module.js:573:30)
    at Object.Module._extensions..js (module.js:584:10)
    at Module.load (module.js:507:32)
    at tryModuleLoad (module.js:470:12)
    at Function.Module._load (module.js:462:3)
    at Function.Module.runMain (module.js:609:10)
    at startup (bootstrap_node.js:158:16)
    at bootstrap_node.js:598:3

------------------------------------------

thread '[ui] ui/deriving-meta-empty-trait-list.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2883:9

---- [ui] ui/enum-size-variance.rs stdout ----

error: test run failed!
status: exit code: 101
command: "/usr/local/bin/node" "/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js" "/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/test/ui/enum-size-variance.stage2-wasm32-unknown-unknown.wasm"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
RuntimeError: unreachable
    at <WASM UNNAMED> (<WASM>[50]+271)
    at <WASM UNNAMED> (<WASM>[189]+76)
    at Object.<anonymous> (/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js:130:20)
    at Module._compile (module.js:573:30)
    at Object.Module._extensions..js (module.js:584:10)
    at Module.load (module.js:507:32)
    at tryModuleLoad (module.js:470:12)
    at Function.Module._load (module.js:462:3)
    at Function.Module.runMain (module.js:609:10)
    at startup (bootstrap_node.js:158:16)
    at bootstrap_node.js:598:3

------------------------------------------

thread '[ui] ui/enum-size-variance.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2883:9

---- [ui] ui/issue-19100.rs stdout ----

error: test run failed!
status: exit code: 101
command: "/usr/local/bin/node" "/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js" "/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/test/ui/issue-19100.stage2-wasm32-unknown-unknown.wasm"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
RuntimeError: unreachable
    at <WASM UNNAMED> (<WASM>[50]+271)
    at <WASM UNNAMED> (<WASM>[189]+76)
    at Object.<anonymous> (/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js:130:20)
    at Module._compile (module.js:573:30)
    at Object.Module._extensions..js (module.js:584:10)
    at Module.load (module.js:507:32)
    at tryModuleLoad (module.js:470:12)
    at Function.Module._load (module.js:462:3)
    at Function.Module.runMain (module.js:609:10)
    at startup (bootstrap_node.js:158:16)
    at bootstrap_node.js:598:3

------------------------------------------

thread '[ui] ui/issue-19100.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2883:9

---- [ui] ui/issue-43806.rs stdout ----

error: test run failed!
status: exit code: 101
command: "/usr/local/bin/node" "/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js" "/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/test/ui/issue-43806.stage2-wasm32-unknown-unknown.wasm"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
RuntimeError: unreachable
    at <WASM UNNAMED> (<WASM>[50]+271)
    at <WASM UNNAMED> (<WASM>[189]+76)
    at Object.<anonymous> (/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js:130:20)
    at Module._compile (module.js:573:30)
    at Object.Module._extensions..js (module.js:584:10)
    at Module.load (module.js:507:32)
    at tryModuleLoad (module.js:470:12)
    at Function.Module._load (module.js:462:3)
    at Function.Module.runMain (module.js:609:10)
    at startup (bootstrap_node.js:158:16)
    at bootstrap_node.js:598:3

------------------------------------------

thread '[ui] ui/issue-43806.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2883:9

---- [ui] ui/nll/borrow-use-issue-46875.rs stdout ----

error: test run failed!
status: exit code: 101
command: "/usr/local/bin/node" "/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js" "/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/test/ui/nll/borrow-use-issue-46875.stage2-wasm32-unknown-unknown.wasm"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
RuntimeError: unreachable
    at <WASM UNNAMED> (<WASM>[49]+271)
    at <WASM UNNAMED> (<WASM>[1]+145)
    at Object.<anonymous> (/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js:130:20)
    at Module._compile (module.js:573:30)
    at Object.Module._extensions..js (module.js:584:10)
    at Module.load (module.js:507:32)
    at tryModuleLoad (module.js:470:12)
    at Function.Module._load (module.js:462:3)
    at Function.Module.runMain (module.js:609:10)
    at startup (bootstrap_node.js:158:16)
    at bootstrap_node.js:598:3

------------------------------------------

thread '[ui] ui/nll/borrow-use-issue-46875.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2883:9

---- [ui] ui/path-lookahead.rs stdout ----

error: test run failed!
status: exit code: 101
command: "/usr/local/bin/node" "/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js" "/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/test/ui/path-lookahead.stage2-wasm32-unknown-unknown.wasm"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
RuntimeError: unreachable
    at <WASM UNNAMED> (<WASM>[50]+271)
    at <WASM UNNAMED> (<WASM>[189]+76)
    at Object.<anonymous> (/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js:130:20)
    at Module._compile (module.js:573:30)
    at Object.Module._extensions..js (module.js:584:10)
    at Module.load (module.js:507:32)
    at tryModuleLoad (module.js:470:12)
    at Function.Module._load (module.js:462:3)
    at Function.Module.runMain (module.js:609:10)
    at startup (bootstrap_node.js:158:16)
    at bootstrap_node.js:598:3

------------------------------------------

thread '[ui] ui/path-lookahead.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2883:9

---- [ui] ui/span/unused-warning-point-at-signature.rs stdout ----

error: test run failed!
status: exit code: 101
command: "/usr/local/bin/node" "/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js" "/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/test/ui/span/unused-warning-point-at-signature.stage2-wasm32-unknown-unknown.wasm"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
RuntimeError: unreachable
    at <WASM UNNAMED> (<WASM>[50]+271)
    at <WASM UNNAMED> (<WASM>[189]+76)
    at Object.<anonymous> (/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js:130:20)
    at Module._compile (module.js:573:30)
    at Object.Module._extensions..js (module.js:584:10)
    at Module.load (module.js:507:32)
    at tryModuleLoad (module.js:470:12)
    at Function.Module._load (module.js:462:3)
    at Function.Module.runMain (module.js:609:10)
    at startup (bootstrap_node.js:158:16)
    at bootstrap_node.js:598:3

------------------------------------------

thread '[ui] ui/span/unused-warning-point-at-signature.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2883:9

---- [ui] ui/test-should-panic-attr.rs stdout ----

error: test run failed!
status: exit code: 101
command: "/usr/local/bin/node" "/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js" "/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/test/ui/test-should-panic-attr.stage2-wasm32-unknown-unknown.wasm"
stdout:
------------------------------------------

------------------------------------------
stderr:
------------------------------------------
RuntimeError: unreachable
    at <WASM UNNAMED> (<WASM>[257]+280)
    at <WASM UNNAMED> (<WASM>[431]+76)
    at Object.<anonymous> (/Volumes/pepa-cloud/dev/rust/src/etc/wasm32-shim.js:130:20)
    at Module._compile (module.js:573:30)
    at Object.Module._extensions..js (module.js:584:10)
    at Module.load (module.js:507:32)
    at tryModuleLoad (module.js:470:12)
    at Function.Module._load (module.js:462:3)
    at Function.Module.runMain (module.js:609:10)
    at startup (bootstrap_node.js:158:16)
    at bootstrap_node.js:598:3

------------------------------------------

thread '[ui] ui/test-should-panic-attr.rs' panicked at 'explicit panic', tools/compiletest/src/runtest.rs:2883:9


failures:
    [ui] ui/deprecated-macro_escape-inner.rs
    [ui] ui/deprecated-macro_escape.rs
    [ui] ui/deriving-meta-empty-trait-list.rs
    [ui] ui/enum-size-variance.rs
    [ui] ui/issue-19100.rs
    [ui] ui/issue-43806.rs
    [ui] ui/nll/borrow-use-issue-46875.rs
    [ui] ui/path-lookahead.rs
    [ui] ui/span/unused-warning-point-at-signature.rs
    [ui] ui/test-should-panic-attr.rs

test result: FAILED. 966 passed; 10 failed; 6 ignored; 0 measured; 0 filtered out

thread 'main' panicked at 'Some tests failed', tools/compiletest/src/main.rs:476:22

However, I'm exteremely not sure that I set up things well.

You can take a look at the change here

@fitzgen
Copy link
Member Author

@fitzgen fitzgen commented Feb 6, 2018

Did you build with debug info? I'd expect that to give function names rather than <WASM>[123]+456.

I was imagining trying to bootstrap rustc with the unix version of wee_alloc -- I'm unsure if rustc will bootstrap with dlmalloc, or any other allocator, when targeting wasm.

@pepyakin
Copy link
Member

@pepyakin pepyakin commented Feb 6, 2018

I was imagining trying to bootstrap rustc with the unix version of wee_alloc

Ahaha, I should have figure it out! Will try : )

Did you build with debug info? I'd expect that to give function names rather than [123]+456.

Nope, im even not sure how to do it. But yeah, AFAIK, debug info + latest node should show mangled names on traps.

@pepyakin
Copy link
Member

@pepyakin pepyakin commented Feb 7, 2018

I'm not sure what this means, but compiling stage1 with stage0 which uses wee_alloc is running for too long (2hrs) on Compiling rustc_tsan v0.0.0 (.../rust/src/librustc_tsan). top reports that 1 core is utilized by 100%.

Can it be just so awfuly slow?

UPD:

CPU utilization is not constant 100% and memory consumption seems to be raising (although, very slowly, from 250 mb to 390 in 1hr)

@fitzgen
Copy link
Member Author

@fitzgen fitzgen commented Feb 7, 2018

I've never actually bootstrapped rustc myself, so I don't know what is a reasonably amount of time...

Maybe try a normal bootstrap and compare?

@fitzgen
Copy link
Member Author

@fitzgen fitzgen commented Feb 7, 2018

Just asked aturon, and he said he expects something closer to half an hour...

Seems like we have some low hanging perf work to do!

@pepyakin
Copy link
Member

@pepyakin pepyakin commented Feb 7, 2018

Yeah usualy it takea around 0.5 to get stage2 with default settings. In this case it takes more that two hours to compile single crate!

@pepyakin
Copy link
Member

@pepyakin pepyakin commented Feb 7, 2018

(it still didnt complete yet)

@fitzgen
Copy link
Member Author

@fitzgen fitzgen commented Feb 7, 2018

(it still didnt complete yet)

lol... this is what happens when there's been zero performance work yet...

@pepyakin
Copy link
Member

@pepyakin pepyakin commented Feb 7, 2018

Aaaaaaand it's done! Sort of

error: Could not compile `core`.

Caused by:
  process didn't exit successfully: `/Volumes/pepa-cloud/dev/rust/build/bootstrap/debug/rustc --crate-name core libcore/lib.rs --error-format json --crate-type lib --emit=dep-info,link -C opt-level=2 -C metadata=807363e85b1c090e -C extra-filename=-807363e85b1c090e --out-dir /Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/stage1-std/x86_64-apple-darwin/release/deps --target x86_64-apple-darwin -L dependency=/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/stage1-std/x86_64-apple-darwin/release/deps -L dependency=/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/stage1-std/release/deps` (signal: 4, SIGILL: illegal instruction)
thread 'main' panicked at 'command did not execute successfully: "/Volumes/pepa-cloud/dev/rust/build/x86_64-apple-darwin/stage0/bin/cargo" "build" "--target" "x86_64-apple-darwin" "-j" "8" "--release" "--features" "panic-unwind jemalloc backtrace" "--manifest-path" "/Volumes/pepa-cloud/dev/rust/src/libstd/Cargo.toml" "--message-format" "json"
expected success, got: exit code: 101', bootstrap/compile.rs:1061:9
note: Run with `RUST_BACKTRACE=1` for a backtrace.
failed to run: /Volumes/pepa-cloud/dev/rust/build/bootstrap/debug/bootstrap build
Build completed unsuccessfully in 4:53:19

(and yes, it turns out it stucked on core crate and not on rustc_tsan as I initially thought)

@fitzgen
Copy link
Member Author

@fitzgen fitzgen commented Feb 7, 2018

Looks like we have some work cut out for us :)

@fitzgen fitzgen changed the title Try bootstrapping rustc with wee_alloc as the global allocator Bootstrapping rustc with wee_alloc as the global allocator should work Feb 7, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants