Skip to content

/rust

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable bootstrapping non-build-machine targets #31884

Merged
merged 14 commits into from Mar 1, 2016

Conversation

Reviewers
No reviews
Assignees

@brson brson

Labels
None yet
Projects
None yet
Milestone
No milestone
5 participants
@alexcrichton @japaric @brson @bors @joerg-krause
@alexcrichton
Copy link
Member

alexcrichton commented Feb 25, 2016

These commits add support to the rustbuild build system to compile non-build-system compilers. For example this will allow creating an ARM compiler on a x86 system. The high level way this works is:

  • The only compiler ever run is the build target compiler. No other compiler is assumed to be runnable.
  • As a result, all output artifacts come from the build compiler.
  • The libs for the stageN cross-compiled compiler will be linked into place after the build target builds them.

This will break the assumption that a compiler never links to anything it didn't itself produce, but it retains the assumption that a compiler only ever links to anything built in the same stage. I believe this means that the stage1 cross-compiled compilers will still be usable.

I tested this by creating an arm-unknown-linux-gnueabihf compiler. So far the linking ended up all working OK (including LLVM being cross compiled), but I haven't been able to run it yet (in QEMU for a raspberry pi). I think that's because my system linker is messed up or something like that (some newer option is assumed when it's not actually there).

Overall, though, this means that rustbuild can compile an arm-unknown-linux-gnueabihf compiler. Ideally we could even start shipping nightlies based on this at some point!
@alexcrichton

This comment has been minimized.

Sign in to view
Copy link
Member Author

alexcrichton commented Feb 25, 2016

r? @brson

@alexcrichton alexcrichton assigned brson Feb 25, 2016

@alexcrichton alexcrichton force-pushed the alexcrichton:no-bootstrap-from-existing branch from cef37ef to 0d7be2e Feb 25, 2016

src/bootstrap/build/native.rs Outdated
@@ -115,6 +115,11 @@ pub fn compiler_rt(build: &Build, target: &str) {
let mode = if build.config.rust_optimize {"Release"} else {"Debug"};
let (dir, build_target, libname) = if target.contains("linux") {
let os = if target.contains("android") {"-android"} else {""};
let arch = if arch == "arm" && target.contains("eabihf") {

This comment has been minimized.

Sign in to view
Copy link
@japaric

japaric Feb 27, 2016

Member

does this cover the armv7-unknown-linux-gnueabihf target?

This comment has been minimized.

Sign in to view
Copy link
@alexcrichton

alexcrichton Feb 28, 2016

Author Member

I... doubt it will!
@japaric

This comment has been minimized.

Sign in to view
Copy link
Member

japaric commented Feb 27, 2016

I've tested this PR by cross compiling a compiler to arm-unknown-linux-gnueabihf. And, then testing that cross compiled rustc/std on an ARMv7 board by running some smoke tests. The smoke tests consisted of: building cargo, cargo testing hyper and cargo testing clap and all of them succeeded. (NOTE: I needed cargo on the ARM target, so I used an unofficial cargo binary from RustBuild. The host was an x86_64 machine running Ubuntu 15.10. The target was an ARMv7 machine running hard-float Ubuntu 15.10)

I found two (minor) issues. @alexcrichton is already aware of them, but to reiterate:

  • We use the flags produced by the host llvm-config --cxxflags to cross compile rustc_llvm. Among these flags there is a -m64 flag that's not recognized by the ARM cross g++ compiler. This flag needs to be filtered out before passing the flags to g++.
  • The cross compiled libcompiler-rt.a doesn't get copied to the build/arm-unknown-linux-gnueabihf/stageN directories.

alexcrichton added some commits Feb 24, 2016

@alexcrichton
rustbuild: Enable bootstrapping new hosts 
This commit fixes a longstanding issue with the makefiles where all host
platforms bootstrap themselves. This commit alters the build logic for the
bootstrap to instead only bootstrap the build triple, and all other compilers
are compiled from that one compiler.

The benefit of this change is that we can cross-compile compilers which cannot
run on the build platform. For example our builders could start creating
`arm-unknown-linux-gnueabihf` compilers.

This reduces the amount of bootstrapping we do, reducing the amount of test
coverage, but overall it should largely just end in faster build times for
multi-host compiles as well as enabling a feature which can't be done today.

cc #5258
90d28ec
@alexcrichton
rustbuild: Only pass RUSTC_FLAGS to target compiles 
These flags aren't applicable to build scripts, and may actuall wreak havoc.
93a3b29
@alexcrichton
rustbuild: Sync changes to Cargo.lock
52ec682
@alexcrichton
rustbuild: Fix compiler-rt build on gnueabihf 
Needs a different target to get built and also we apparently need to appease the
C++ compiler somehow.
a707a61
@alexcrichton
rustbuild: Enable cross-compiling LLVM 
Currently all multi-host builds assume the the build platform can run the
`llvm-config` binary generated for each host platform we're creating a compiler
for. Unfortunately this assumption isn't always true when cross compiling, so we
need to handle this case.

This commit alters the build script of `rustc_llvm` to understand when it's
running an `llvm-config` which is different than the platform we're targeting for.
5266406
@alexcrichton
rustbuild: Fix a copy/paste error 
Fixes `--step librustc`
189827b
@alexcrichton
rustbuild: Move assembling rustc to its own step 
Right now it's implicitly done as part of building the compiler, but this was
intended to be a standalone step to ensure we tracked what built what.
ed3d46d
@alexcrichton
rustbuild: Compile with the build compiler 
This switches the defaults to ensure that everything is built with the build
compiler rather than the host compiler itself (which we're not guaranteed to be
able to run)
5abcc78
@alexcrichton
rustbuild: Document what steps are
0677387
@alexcrichton
rustbuild: Add steps for linking a sysroot 
When cross compiling for a new host, we can't actually run the host compiler to
generate its own libs. In theory, however, all stage2 compilers (for any host)
will produce the same libraries, so we just require the build compiler to
produce the necessary host libraries and then we link those into place.
e9a897c
@alexcrichton
rustbuild: Remove extra rustc flags 
These should all no longer be necessary as they've been folded into the
compiler.
506522f
@alexcrichton
rustbuild: Relax assertions about stage0 
This allows bootstrapping new platforms immediately in stage0
93a9ab1
@alexcrichton
rustbuild: Update nightly date 
Also fix a bug where we didn't clean out previous nightlies
17ae752
@alexcrichton
rustbuild: Update dependencies 
* Fixes a warning with libc
* Brings in some new flag updates for various platforms through gcc-rs
* Otherwise routine updates here/there
Loading status checks…
15b4a8c

@alexcrichton alexcrichton force-pushed the alexcrichton:no-bootstrap-from-existing branch from 86d83be to 15b4a8c Feb 28, 2016

@alexcrichton

This comment has been minimized.

Sign in to view
Copy link
Member Author

alexcrichton commented Feb 28, 2016

Thanks for testing this out @japaric! I've amended the relevant commits and those issues should all be fixed.
@brson

This comment has been minimized.

Sign in to view
Copy link
Contributor

brson commented Feb 29, 2016

@bors r+
@bors

This comment has been minimized.

Sign in to view
Copy link
Contributor

bors commented Feb 29, 2016

📌 Commit 15b4a8c has been approved by brson
@joerg-krause

This comment has been minimized.

Sign in to view
Copy link
Contributor

joerg-krause commented Feb 29, 2016

How do you exactly use this feature? Can you give an example on howto build for the ARM target you mentioned?
@japaric

This comment has been minimized.

Sign in to view
Copy link
Member

japaric commented Feb 29, 2016

How do you exactly use this feature? Can you give an example on howto build for the ARM target you mentioned?

Pass --host=$triple and --enable-rustbuild to the configure script before calling make. Something like this:

mkdir build
cd build
../configure --enable-rustbuild --host=arm-unknown-linux-gnueabihf
make

bors added a commit that referenced this pull request Feb 29, 2016

@bors
Auto merge of #31884 - alexcrichton:no-bootstrap-from-existing, r=brson 
These commits add support to the rustbuild build system to compile non-build-system compilers. For example this will allow creating an ARM compiler on a x86 system. The high level way this works is:

* The only compiler ever run is the build target compiler. No other compiler is assumed to be runnable.
* As a result, all output artifacts come from the build compiler.
* The libs for the stageN cross-compiled compiler will be linked into place after the build target builds them.

This will break the assumption that a compiler never links to anything it didn't itself produce, but it retains the assumption that a compiler only ever links to anything built in the same stage. I believe this means that the stage1 cross-compiled compilers will still be usable.

I tested this by creating an `arm-unknown-linux-gnueabihf` compiler. So far the linking ended up all working OK (including LLVM being cross compiled), but I haven't been able to run it yet (in QEMU for a raspberry pi). I think that's because my system linker is messed up or something like that (some newer option is assumed when it's not actually there).

Overall, though, this means that rustbuild can compile an arm-unknown-linux-gnueabihf compiler. Ideally we could even start shipping nightlies based on this at some point!
5a0308a
@bors

This comment has been minimized.

Sign in to view
Copy link
Contributor

bors commented Feb 29, 2016

⌛️ Testing commit 15b4a8c with merge 5a0308a...
@bors

@bors bors merged commit 15b4a8c into rust-lang:master Mar 1, 2016
2 checks passed

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
homu Test successful
Details

@alexcrichton alexcrichton deleted the alexcrichton:no-bootstrap-from-existing branch Mar 2, 2016

@alexcrichton alexcrichton referenced this pull request Mar 2, 2016

Closed

Tracking issue for rustbuild, the alternate build system for rustc #31590

42 of 48 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.