[WIP] Use rustup to manage toolchains #17927

shinglyu · 2017-08-01T03:51:18Z

[DO NOT MERGE]
This is an early prototype of using rustup to replace our rustc/cargo download logic. I'd like to collect some feedback before I start working on other platforms.

I'd also love to hear your opinion on how to approach the following issues:

Windows/OSX build: any special rustup tricks?
Android build: Should we use rustup at all?
Cross compiling: What kind of use cases are there?

ping @metajack

./mach build -d does not report any errors
./mach test-tidy does not report any errors
These changes fix #11361 .

There are tests for these changes OR
These changes do not require tests because _____

This change is


        [WIP] Use rustup to manage toolchains

highfive · 2017-08-01T03:51:23Z

Heads up! This PR modifies the following files:

@wafflespeanut: python/servo/command_base.py, python/servo/bootstrap_commands.py

SimonSapin · 2017-08-01T06:34:11Z

rust-toolchain should replace rust-commit-hash. If we have both they’ll get out of sync. We can parse e.g. https://static.rust-lang.org/dist/2017-08-01/channel-rust-nightly.toml and https://api.github.com/repos/rust-lang/rust/commits/37c7d0ebb to go from a nightly date to a full SHA1 hash.
I’d like to have a solution or work-around for rust-lang/rustup#1099 before this is merged, so that compilation times do not regress by 15~50%.
For nigthly vs stable Rust, perhaps it’d work out to have both use ./rust-toolchain and the root of the repository and ./ports/geckolib/rust-toolchain? The ./mach build-geckolib command already uses with cd(path.join("ports", "geckolib")).

shinglyu · 2017-08-02T02:46:01Z

@SimonSapin: Thanks for the first two comments! As for the nightly vs stable issue, is geckolib the only case we use stable rust? If so would it be safe to remove (at least part) of the stable flags and use rust-toolchain to control this instead?

shinglyu · 2017-08-02T02:52:56Z

Also if not using master-alt will hurt the performance badly, I can land the rustup feature prefed-off. Because the only people who need this eagerly is the people working on embedding, so I guess pref-on rustup for them is not a big deal.

SimonSapin · 2017-08-02T08:02:28Z

Yes, geckolib is the only case where we use stable Rust. "Stable Rust version" in the context of Servo really means "minimum version that Firefox wants to support". (Search for "Mozilla-central can use" in https://wiki.mozilla.org/Rust_Update_Policy_for_Firefox.)

And, indeed, LLVM assertions are already disabled in official stable channel builds, so "alt" toolchains are not relevant there.

I think it’s fine to require that rustup is available for building Servo. Eventually I’d like to entirely remove our code for downloading tarballs and setting $PATH environment variables, and leave that job to rustup. I’m not sure what’s the best way to handle alt toolchains until they’re natively supported by rustup, though.

metajack · 2017-08-02T08:08:21Z

@shinglyu I believe the plan is that rustup will have access to alt toolchains soon.

shinglyu · 2017-08-03T03:10:43Z

OK! I'll keep an eye on the alt-toolchains support for rustup

paulrouget · 2017-08-31T05:34:18Z

I believe the plan is that rustup will have access to alt toolchains soon.

Is there any issue/PR we can look at?

SimonSapin · 2017-08-31T06:49:25Z

rust-lang/rustup#1099 and https://internals.rust-lang.org/t/disabling-llvm-assertions-in-nightly-builds/5388 still need a decision on what’s the plan exactly.


        Switch back to pinning Rust by Nightly date instead of commit hash…

… this time using a `rust-toolchain` file compatible with rustup: https://github.com/rust-lang-nursery/rustup.rs/#the-toolchain-file And upgrade to rustc 1.21.0-nightly (c11f689d2 2017-08-29) ---- Now if both `system-rust` and `system-cargo` are set to `true` in `.servobuild`’s `[tools]` section, and the corresponding `rustc` and `cargo` binaries are in fact rustup’s wrappers, then rustup will use the correct version based on `rust-toolchain`. CC #11361 Unlike #17927, this does not make mach use rustup directly.


        Switch back to pinning Rust by Nightly date instead of commit hash…

… this time using a `rust-toolchain` file compatible with rustup: https://github.com/rust-lang-nursery/rustup.rs/#the-toolchain-file And upgrade to rustc 1.21.0-nightly (c11f689d2 2017-08-29) ---- Now if both `system-rust` and `system-cargo` are set to `true` in `.servobuild`’s `[tools]` section, and the corresponding `rustc` and `cargo` binaries are in fact rustup’s wrappers, then rustup will use the correct version based on `rust-toolchain`. CC #11361 Unlike #17927, this does not make mach use rustup directly.


        Auto merge of #18325 - servo:rustup-toolchain, r=nox

Switch back to pinning Rust by Nightly date instead of commit hash… … this time using a `rust-toolchain` file compatible with rustup: https://github.com/rust-lang-nursery/rustup.rs/#the-toolchain-file And upgrade to rustc 1.21.0-nightly (c11f689d2 2017-08-29) ---- Now if both `system-rust` and `system-cargo` are set to `true` in `.servobuild`’s `[tools]` section, and the corresponding `rustc` and `cargo` binaries are in fact rustup’s wrappers, then rustup will use the correct version based on `rust-toolchain`. CC #11361 Unlike #17927, this does not make mach use rustup directly. That should wait until rust-lang/rustup#1099 is fixed.  --- This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/servo/servo/18325)

bors-servo · 2017-08-31T14:43:14Z

☔ The latest upstream changes (presumably #18325) made this pull request unmergeable. Please resolve the merge conflicts.


        servo: Merge #18325 - Switch back to pinning Rust by Nightly date ins…

…tead of commit hash… (from servo:rustup-toolchain); r=nox … this time using a `rust-toolchain` file compatible with rustup: https://github.com/rust-lang-nursery/rustup.rs/#the-toolchain-file And upgrade to rustc 1.21.0-nightly (c11f689d2 2017-08-29) ---- Now if both `system-rust` and `system-cargo` are set to `true` in `.servobuild`’s `[tools]` section, and the corresponding `rustc` and `cargo` binaries are in fact rustup’s wrappers, then rustup will use the correct version based on `rust-toolchain`. CC servo/servo#11361 Unlike servo/servo#17927, this does not make mach use rustup directly. That should wait until rust-lang/rustup#1099 is fixed. Source-Repo: https://github.com/servo/servo Source-Revision: c4800a6c83e6fdabaf7c4eff70a24487d16f18ff --HG-- extra : subtree_source : https%3A//hg.mozilla.org/projects/converted-servo-linear extra : subtree_revision : e3d41159bc4670ea990630e5342fe9b663a79fe5


        servo: Merge #18325 - Switch back to pinning Rust by Nightly date ins…

…tead of commit hash… (from servo:rustup-toolchain); r=nox … this time using a `rust-toolchain` file compatible with rustup: https://github.com/rust-lang-nursery/rustup.rs/#the-toolchain-file And upgrade to rustc 1.21.0-nightly (c11f689d2 2017-08-29) ---- Now if both `system-rust` and `system-cargo` are set to `true` in `.servobuild`’s `[tools]` section, and the corresponding `rustc` and `cargo` binaries are in fact rustup’s wrappers, then rustup will use the correct version based on `rust-toolchain`. CC servo/servo#11361 Unlike servo/servo#17927, this does not make mach use rustup directly. That should wait until rust-lang/rustup#1099 is fixed. Source-Repo: https://github.com/servo/servo Source-Revision: c4800a6c83e6fdabaf7c4eff70a24487d16f18ff


        servo: Merge #18325 - Switch back to pinning Rust by Nightly date ins…

…tead of commit hash… (from servo:rustup-toolchain); r=nox … this time using a `rust-toolchain` file compatible with rustup: https://github.com/rust-lang-nursery/rustup.rs/#the-toolchain-file And upgrade to rustc 1.21.0-nightly (c11f689d2 2017-08-29) ---- Now if both `system-rust` and `system-cargo` are set to `true` in `.servobuild`’s `[tools]` section, and the corresponding `rustc` and `cargo` binaries are in fact rustup’s wrappers, then rustup will use the correct version based on `rust-toolchain`. CC servo/servo#11361 Unlike servo/servo#17927, this does not make mach use rustup directly. That should wait until rust-lang/rustup#1099 is fixed. Source-Repo: https://github.com/servo/servo Source-Revision: c4800a6c83e6fdabaf7c4eff70a24487d16f18ff

SimonSapin · 2018-01-10T19:45:37Z

Done in #19395.


        servo: Merge #18325 - Switch back to pinning Rust by Nightly date ins…

…tead of commit hash… (from servo:rustup-toolchain); r=nox … this time using a `rust-toolchain` file compatible with rustup: https://github.com/rust-lang-nursery/rustup.rs/#the-toolchain-file And upgrade to rustc 1.21.0-nightly (c11f689d2 2017-08-29) ---- Now if both `system-rust` and `system-cargo` are set to `true` in `.servobuild`’s `[tools]` section, and the corresponding `rustc` and `cargo` binaries are in fact rustup’s wrappers, then rustup will use the correct version based on `rust-toolchain`. CC servo/servo#11361 Unlike servo/servo#17927, this does not make mach use rustup directly. That should wait until rust-lang/rustup#1099 is fixed. Source-Repo: https://github.com/servo/servo Source-Revision: c4800a6c83e6fdabaf7c4eff70a24487d16f18ff UltraBlame original commit: 5df6f8fe6c50631fe00f47b69d16b8d555d90503


        servo: Merge #18325 - Switch back to pinning Rust by Nightly date ins…

…tead of commit hash… (from servo:rustup-toolchain); r=nox … this time using a `rust-toolchain` file compatible with rustup: https://github.com/rust-lang-nursery/rustup.rs/#the-toolchain-file And upgrade to rustc 1.21.0-nightly (c11f689d2 2017-08-29) ---- Now if both `system-rust` and `system-cargo` are set to `true` in `.servobuild`’s `[tools]` section, and the corresponding `rustc` and `cargo` binaries are in fact rustup’s wrappers, then rustup will use the correct version based on `rust-toolchain`. CC servo/servo#11361 Unlike servo/servo#17927, this does not make mach use rustup directly. That should wait until rust-lang/rustup#1099 is fixed. Source-Repo: https://github.com/servo/servo Source-Revision: c4800a6c83e6fdabaf7c4eff70a24487d16f18ff UltraBlame original commit: 5df6f8fe6c50631fe00f47b69d16b8d555d90503


        servo: Merge #18325 - Switch back to pinning Rust by Nightly date ins…

…tead of commit hash… (from servo:rustup-toolchain); r=nox … this time using a `rust-toolchain` file compatible with rustup: https://github.com/rust-lang-nursery/rustup.rs/#the-toolchain-file And upgrade to rustc 1.21.0-nightly (c11f689d2 2017-08-29) ---- Now if both `system-rust` and `system-cargo` are set to `true` in `.servobuild`’s `[tools]` section, and the corresponding `rustc` and `cargo` binaries are in fact rustup’s wrappers, then rustup will use the correct version based on `rust-toolchain`. CC servo/servo#11361 Unlike servo/servo#17927, this does not make mach use rustup directly. That should wait until rust-lang/rustup#1099 is fixed. Source-Repo: https://github.com/servo/servo Source-Revision: c4800a6c83e6fdabaf7c4eff70a24487d16f18ff UltraBlame original commit: 5df6f8fe6c50631fe00f47b69d16b8d555d90503

[WIP] Use rustup to manage toolchains

7b67fa3

highfive assigned wafflespeanut Aug 1, 2017

highfive added the S-awaiting-review label Aug 1, 2017

jdm assigned metajack and unassigned wafflespeanut Aug 1, 2017

SimonSapin mentioned this pull request Aug 31, 2017

Switch back to pinning Rust by Nightly date instead of commit hash… #18325

Merged

highfive added the S-needs-rebase label Aug 31, 2017

SimonSapin closed this Jan 10, 2018

servo / servo

[WIP] Use rustup to manage toolchains #17927

[WIP] Use rustup to manage toolchains #17927

shinglyu commented Aug 1, 2017 •

edited by SimonSapin

highfive commented Aug 1, 2017

SimonSapin commented Aug 1, 2017

shinglyu commented Aug 2, 2017

shinglyu commented Aug 2, 2017

SimonSapin commented Aug 2, 2017

metajack commented Aug 2, 2017

shinglyu commented Aug 3, 2017

paulrouget commented Aug 31, 2017

SimonSapin commented Aug 31, 2017

bors-servo commented Aug 31, 2017

SimonSapin commented Jan 10, 2018

servo / servo

Join GitHub today

[WIP] Use rustup to manage toolchains #17927

[WIP] Use rustup to manage toolchains #17927

Conversation

shinglyu commented Aug 1, 2017 • edited by SimonSapin

highfive commented Aug 1, 2017

SimonSapin commented Aug 1, 2017

shinglyu commented Aug 2, 2017

shinglyu commented Aug 2, 2017

SimonSapin commented Aug 2, 2017

metajack commented Aug 2, 2017

shinglyu commented Aug 3, 2017

paulrouget commented Aug 31, 2017

SimonSapin commented Aug 31, 2017

bors-servo commented Aug 31, 2017

SimonSapin commented Jan 10, 2018

shinglyu commented Aug 1, 2017 •

edited by SimonSapin