New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

mk: Bootstrap from stable instead of snapshots #32942

Merged
merged 1 commit into from Apr 20, 2016

Conversation

Projects
None yet
8 participants
@alexcrichton
Member

alexcrichton commented Apr 13, 2016

This commit removes all infrastructure from the repository for our so-called
snapshots to instead bootstrap the compiler from stable releases. Bootstrapping
from a previously stable release is a long-desired feature of distros because
they're not fans of downloading binary stage0 blobs from us. Additionally, this
makes our own CI easier as we can decommission all of the snapshot builders and
start having a regular cadence to when we update the stage0 compiler.

A new src/etc/get-stage0.py script was added which shares some code with
src/bootstrap/bootstrap.py to read a new file, src/stage0.txt, which lists
the current stage0 compiler as well as cargo that we bootstrap from. This script
will download the relevant rustc package an unpack it into $target/stage0 as
we do today.

One problem of bootstrapping from stable releases is that we're not able to
compile unstable code (e.g. all the #![feature] directives in libcore/libstd).
To overcome this we employ two strategies:

  • The bootstrap key of the previous compiler is hardcoded into src/stage0.txt
    (enabled as a result of #32731) and exported by the build system. This enables
    nightly features in the compiler we download.
  • The standard library and compiler are pinned to a specific stage0, which
    doesn't change, so we're guaranteed that we'll continue compiling as we start
    from a known fixed source.

The process for making a release will also need to be tweaked now to continue to
cadence of bootstrapping from the previous release. This process looks like:

  1. Merge beta to stable
  2. Produce a new stable compiler.
  3. Change master to bootstrap from this new stable compiler.
  4. Merge master to beta
  5. Produce a new beta compiler
  6. Change master to bootstrap from this new beta compiler.

Step 3 above should involve very few changes as master was previously
bootstrapping from beta which is the same as stable at that point in time.
Step 6, however, is where we benefit from removing lots of #[cfg(stage0)] and
get to use new features. This also shouldn't slow the release too much as steps
1-5 requires little work other than waiting and step 6 just needs to happen at
some point during a release cycle, it's not time sensitive.

Closes #29555
Closes #29557

@rust-highfive

This comment has been minimized.

Show comment
Hide comment
@rust-highfive

rust-highfive Apr 13, 2016

Collaborator

r? @aturon

(rust_highfive has picked a reviewer for you, use r? to override)

Collaborator

rust-highfive commented Apr 13, 2016

r? @aturon

(rust_highfive has picked a reviewer for you, use r? to override)

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Apr 13, 2016

Member

r? @brson

This commit moves us to bootstrapping from the recently-produced 1.9 beta compiler as a proof-of-concept for both the makefiles and rustbuild.

cc @dhuseby, @mneumann, @semarie -- this will unfortunately break BSD bootstraps for the near future. We don't have stable releases for platforms like OpenBSD, Bitrig, or DragonFly (we do for FreeBSD/NetBSD now), but we'd have to hook up some more automation to get this working.

In the interim we can perhaps produce releases after the fact for these platforms, but it would be best if we could figure out how to cross-compile these platforms from Linux (like FreeBSD and NetBSD are doing). That way we could hook everything up to our automation, make sure releases/nightlies are made, everything keeps building, etc.

Member

alexcrichton commented Apr 13, 2016

r? @brson

This commit moves us to bootstrapping from the recently-produced 1.9 beta compiler as a proof-of-concept for both the makefiles and rustbuild.

cc @dhuseby, @mneumann, @semarie -- this will unfortunately break BSD bootstraps for the near future. We don't have stable releases for platforms like OpenBSD, Bitrig, or DragonFly (we do for FreeBSD/NetBSD now), but we'd have to hook up some more automation to get this working.

In the interim we can perhaps produce releases after the fact for these platforms, but it would be best if we could figure out how to cross-compile these platforms from Linux (like FreeBSD and NetBSD are doing). That way we could hook everything up to our automation, make sure releases/nightlies are made, everything keeps building, etc.

@rust-highfive rust-highfive assigned brson and unassigned aturon Apr 13, 2016

@brson

This comment has been minimized.

Show comment
Hide comment
@brson

brson Apr 13, 2016

Contributor

r=me but we should probably hear from the *BSD maintainers before going forward.

Contributor

brson commented Apr 13, 2016

r=me but we should probably hear from the *BSD maintainers before going forward.

@dhuseby

This comment has been minimized.

Show comment
Hide comment
@dhuseby

dhuseby Apr 14, 2016

@alexcrichton so we should all focus on getting cross-compiling from linux to work? that's trivial for netbsd and probably works for freebsd, but openbsd is likely much harder due to the use of older/custom versions of the gnu toolchain. I know that bitrig will be very difficult as they rely solely on clang and libc++. Convincing Linux to only use that toolchain and libraries is not trivial. The last time I figure it out it looked like this:

compile:

clang++ -std=c++11 -nostdlib -mstackrealign -g -I/usr/include/c++/v1/ -I/usr/include/libcxxabi -I/usr/lib/llvm-3.4/include  -DNDEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -g -O2 -fomit-frame-pointer -fvisibility-inlines-hidden -fno-exceptions -fPIC -Woverloaded-virtual -Wcast-qual -c test2.cc

link:

/usr/bin/ld -z relro --hash-style=gnu --build-id --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o test2 /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crt1.o /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crti.o /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/crtbegin.o -L/usr/lib/llvm-3.4/lib -L/usr/lib/x86_64-linux-gnu/ -L/lib64 -L/lib -L/usr/lib test2.o -lc++ -lc++abi -lunwind -lc -lpthread -lffi -ltinfo -ldl -lm /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/crtend.o /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crtn.o

ldd:

        linux-vdso.so.1 =>  (0x00007fff5d7fe000)
        libc++.so.1 => /usr/lib/x86_64-linux-gnu/libc++.so.1 (0x00007fac64095000)
        libc++abi.so.1 => /usr/lib/x86_64-linux-gnu/libc++abi.so.1 (0x00007fac63e46000)
        libunwind.so.8 => /usr/lib/x86_64-linux-gnu/libunwind.so.8 (0x00007fac63c2a000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fac63864000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fac63646000)
        libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007fac6343d000)
        libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007fac63214000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fac63010000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fac62d09000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fac62b01000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fac643ad000)
        liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007fac628df000)

dhuseby commented Apr 14, 2016

@alexcrichton so we should all focus on getting cross-compiling from linux to work? that's trivial for netbsd and probably works for freebsd, but openbsd is likely much harder due to the use of older/custom versions of the gnu toolchain. I know that bitrig will be very difficult as they rely solely on clang and libc++. Convincing Linux to only use that toolchain and libraries is not trivial. The last time I figure it out it looked like this:

compile:

clang++ -std=c++11 -nostdlib -mstackrealign -g -I/usr/include/c++/v1/ -I/usr/include/libcxxabi -I/usr/lib/llvm-3.4/include  -DNDEBUG -D_GNU_SOURCE -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -g -O2 -fomit-frame-pointer -fvisibility-inlines-hidden -fno-exceptions -fPIC -Woverloaded-virtual -Wcast-qual -c test2.cc

link:

/usr/bin/ld -z relro --hash-style=gnu --build-id --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 -o test2 /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crt1.o /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crti.o /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/crtbegin.o -L/usr/lib/llvm-3.4/lib -L/usr/lib/x86_64-linux-gnu/ -L/lib64 -L/lib -L/usr/lib test2.o -lc++ -lc++abi -lunwind -lc -lpthread -lffi -ltinfo -ldl -lm /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/crtend.o /usr/bin/../lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu/crtn.o

ldd:

        linux-vdso.so.1 =>  (0x00007fff5d7fe000)
        libc++.so.1 => /usr/lib/x86_64-linux-gnu/libc++.so.1 (0x00007fac64095000)
        libc++abi.so.1 => /usr/lib/x86_64-linux-gnu/libc++abi.so.1 (0x00007fac63e46000)
        libunwind.so.8 => /usr/lib/x86_64-linux-gnu/libunwind.so.8 (0x00007fac63c2a000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fac63864000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fac63646000)
        libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007fac6343d000)
        libtinfo.so.5 => /lib/x86_64-linux-gnu/libtinfo.so.5 (0x00007fac63214000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fac63010000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fac62d09000)
        librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fac62b01000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fac643ad000)
        liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007fac628df000)
@dhuseby

This comment has been minimized.

Show comment
Hide comment
@dhuseby

dhuseby Apr 14, 2016

@alexcrichton I'm all for figuring this out as the CI will be much more uniform and cleaner.

dhuseby commented Apr 14, 2016

@alexcrichton I'm all for figuring this out as the CI will be much more uniform and cleaner.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Apr 14, 2016

Member

@alexcrichton so we should all focus on getting cross-compiling from linux to work?

For now, yeah, that's our best option for keeping these platforms going. We can very easily set up and manage bots to cross-compile to each BSD, but running dedicated BSD hardware is a much bigger headache for us we aren't willing to stomach just yet.

that's trivial for netbsd and probably works for freebsd

Indeed! We've got nightlies/betas for both of them :)

but openbsd is likely much harder due to the use of older/custom versions of the gnu toolchain.

Yeah this is kinda unfortunate :(. There may not be any way to cross from Linux in the long run.


In general this doesn't necessarily mean "game over" for the BSDs, it may just mean that we periodically upload a new beta for each BSD. We probably don't want to push stable releases just yet unless we've got "more confidence" in them, but we could also perhaps do that as well!

In essence this'll just mean that new BSD "snapshots" will need to happen on a 6-week cadence

Member

alexcrichton commented Apr 14, 2016

@alexcrichton so we should all focus on getting cross-compiling from linux to work?

For now, yeah, that's our best option for keeping these platforms going. We can very easily set up and manage bots to cross-compile to each BSD, but running dedicated BSD hardware is a much bigger headache for us we aren't willing to stomach just yet.

that's trivial for netbsd and probably works for freebsd

Indeed! We've got nightlies/betas for both of them :)

but openbsd is likely much harder due to the use of older/custom versions of the gnu toolchain.

Yeah this is kinda unfortunate :(. There may not be any way to cross from Linux in the long run.


In general this doesn't necessarily mean "game over" for the BSDs, it may just mean that we periodically upload a new beta for each BSD. We probably don't want to push stable releases just yet unless we've got "more confidence" in them, but we could also perhaps do that as well!

In essence this'll just mean that new BSD "snapshots" will need to happen on a 6-week cadence

@mneumann

This comment has been minimized.

Show comment
Hide comment
@mneumann

mneumann Apr 14, 2016

Contributor

@alexcrichton I don't see any problems with this change for DragonFly. We are already using FreeBSD's ports infrastructure for building rustc (with some minor patches) and I think the FreeBSD ports maintainers will do all the work for us :)

Contributor

mneumann commented Apr 14, 2016

@alexcrichton I don't see any problems with this change for DragonFly. We are already using FreeBSD's ports infrastructure for building rustc (with some minor patches) and I think the FreeBSD ports maintainers will do all the work for us :)

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Apr 14, 2016

Member

Thanks for the update @mneumann! Ok, @dhuseby so you think it's ok to merge this and we can figure out perhaps making beta releases for these other platforms afterwards?

Member

alexcrichton commented Apr 14, 2016

Thanks for the update @mneumann! Ok, @dhuseby so you think it's ok to merge this and we can figure out perhaps making beta releases for these other platforms afterwards?

@bors

This comment has been minimized.

Show comment
Hide comment
@bors

bors Apr 15, 2016

Contributor

☔️ The latest upstream changes (presumably #32960) made this pull request unmergeable. Please resolve the merge conflicts.

Contributor

bors commented Apr 15, 2016

☔️ The latest upstream changes (presumably #32960) made this pull request unmergeable. Please resolve the merge conflicts.

@semarie

This comment has been minimized.

Show comment
Hide comment
@semarie

semarie Apr 15, 2016

Contributor

@alexcrichton if I correctly understand, for openbsd it will be required to generate a -beta release each 6 weeks (if not crosscompile possibility) ? for me, it won't be a big problem: I could generate it for upload.

does snap3 buildbots will be converted to beta buildbots ?

Contributor

semarie commented Apr 15, 2016

@alexcrichton if I correctly understand, for openbsd it will be required to generate a -beta release each 6 weeks (if not crosscompile possibility) ? for me, it won't be a big problem: I could generate it for upload.

does snap3 buildbots will be converted to beta buildbots ?

@bors

This comment has been minimized.

Show comment
Hide comment
@bors

bors Apr 15, 2016

Contributor

☔️ The latest upstream changes (presumably #32895) made this pull request unmergeable. Please resolve the merge conflicts.

Contributor

bors commented Apr 15, 2016

☔️ The latest upstream changes (presumably #32895) made this pull request unmergeable. Please resolve the merge conflicts.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Apr 15, 2016

Member

@semarie yeah if we want to keep nightly OpenBSD compiling we'll basically need to simulate a beta release for these platforms whenever a beta is built (this includes beta updates as well if we switch to bootstrapping from them).

We'll be retiring the snap3 buildbots and we do indeed have a suite of beta buildbots. We could add an OpenBSD one there, but I'm not sure the snap3 bots have ever produced an OpenBSD snapshot successfully so we may not have much luck with the beta ones either :(

(this is where cross-compiling to OpenBSD is the easiest route for us)

Member

alexcrichton commented Apr 15, 2016

@semarie yeah if we want to keep nightly OpenBSD compiling we'll basically need to simulate a beta release for these platforms whenever a beta is built (this includes beta updates as well if we switch to bootstrapping from them).

We'll be retiring the snap3 buildbots and we do indeed have a suite of beta buildbots. We could add an OpenBSD one there, but I'm not sure the snap3 bots have ever produced an OpenBSD snapshot successfully so we may not have much luck with the beta ones either :(

(this is where cross-compiling to OpenBSD is the easiest route for us)

@bors

This comment has been minimized.

Show comment
Hide comment
@bors

bors Apr 19, 2016

Contributor

☔️ The latest upstream changes (presumably #32755) made this pull request unmergeable. Please resolve the merge conflicts.

Contributor

bors commented Apr 19, 2016

☔️ The latest upstream changes (presumably #32755) made this pull request unmergeable. Please resolve the merge conflicts.

@brson

This comment has been minimized.

Show comment
Hide comment
@brson

brson Apr 19, 2016

Contributor

@bors r+

Contributor

brson commented Apr 19, 2016

@bors r+

@bors

This comment has been minimized.

Show comment
Hide comment
@bors

bors Apr 19, 2016

Contributor

📌 Commit d291511 has been approved by brson

Contributor

bors commented Apr 19, 2016

📌 Commit d291511 has been approved by brson

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Apr 19, 2016

Member

@bors: r-

Gah just realized the travis error is probably legit.

Member

alexcrichton commented Apr 19, 2016

@bors: r-

Gah just realized the travis error is probably legit.

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
Member

alexcrichton commented Apr 19, 2016

@bors: r=brson 7a3ac97

@bors

This comment has been minimized.

Show comment
Hide comment
@bors

bors Apr 19, 2016

Contributor

⌛️ Testing commit 7a3ac97 with merge 04e9d72...

Contributor

bors commented Apr 19, 2016

⌛️ Testing commit 7a3ac97 with merge 04e9d72...

bors added a commit that referenced this pull request Apr 19, 2016

Auto merge of #32942 - alexcrichton:bootstrap-from-previous, r=brson
mk: Bootstrap from stable instead of snapshots

This commit removes all infrastructure from the repository for our so-called
snapshots to instead bootstrap the compiler from stable releases. Bootstrapping
from a previously stable release is a long-desired feature of distros because
they're not fans of downloading binary stage0 blobs from us. Additionally, this
makes our own CI easier as we can decommission all of the snapshot builders and
start having a regular cadence to when we update the stage0 compiler.

A new `src/etc/get-stage0.py` script was added which shares some code with
`src/bootstrap/bootstrap.py` to read a new file, `src/stage0.txt`, which lists
the current stage0 compiler as well as cargo that we bootstrap from. This script
will download the relevant `rustc` package an unpack it into `$target/stage0` as
we do today.

One problem of bootstrapping from stable releases is that we're not able to
compile unstable code (e.g. all the `#![feature]` directives in libcore/libstd).
To overcome this we employ two strategies:

* The bootstrap key of the previous compiler is hardcoded into `src/stage0.txt`
  (enabled as a result of #32731) and exported by the build system. This enables
  nightly features in the compiler we download.
* The standard library and compiler are pinned to a specific stage0, which
  doesn't change, so we're guaranteed that we'll continue compiling as we start
  from a known fixed source.

The process for making a release will also need to be tweaked now to continue to
cadence of bootstrapping from the previous release. This process looks like:

1. Merge `beta` to `stable`
2. Produce a new stable compiler.
3. Change `master` to bootstrap from this new stable compiler.
4. Merge `master` to `beta`
5. Produce a new beta compiler
6. Change `master` to bootstrap from this new beta compiler.

Step 3 above should involve very few changes as `master` was previously
bootstrapping from `beta` which is the same as `stable` at that point in time.
Step 6, however, is where we benefit from removing lots of `#[cfg(stage0)]` and
get to use new features. This also shouldn't slow the release too much as steps
1-5 requires little work other than waiting and step 6 just needs to happen at
some point during a release cycle, it's not time sensitive.

Closes #29555
Closes #29557
@bors

This comment has been minimized.

Show comment
Hide comment
@bors

bors Apr 19, 2016

Contributor

💔 Test failed - auto-linux-64-x-freebsd

Contributor

bors commented Apr 19, 2016

💔 Test failed - auto-linux-64-x-freebsd

mk: Bootstrap from stable instead of snapshots
This commit removes all infrastructure from the repository for our so-called
snapshots to instead bootstrap the compiler from stable releases. Bootstrapping
from a previously stable release is a long-desired feature of distros because
they're not fans of downloading binary stage0 blobs from us. Additionally, this
makes our own CI easier as we can decommission all of the snapshot builders and
start having a regular cadence to when we update the stage0 compiler.

A new `src/etc/get-stage0.py` script was added which shares some code with
`src/bootstrap/bootstrap.py` to read a new file, `src/stage0.txt`, which lists
the current stage0 compiler as well as cargo that we bootstrap from. This script
will download the relevant `rustc` package an unpack it into `$target/stage0` as
we do today.

One problem of bootstrapping from stable releases is that we're not able to
compile unstable code (e.g. all the `#![feature]` directives in libcore/libstd).
To overcome this we employ two strategies:

* The bootstrap key of the previous compiler is hardcoded into `src/stage0.txt`
  (enabled as a result of #32731) and exported by the build system. This enables
  nightly features in the compiler we download.
* The standard library and compiler are pinned to a specific stage0, which
  doesn't change, so we're guaranteed that we'll continue compiling as we start
  from a known fixed source.

The process for making a release will also need to be tweaked now to continue to
cadence of bootstrapping from the previous release. This process looks like:

1. Merge `beta` to `stable`
2. Produce a new stable compiler.
3. Change `master` to bootstrap from this new stable compiler.
4. Merge `master` to `beta`
5. Produce a new beta compiler
6. Change `master` to bootstrap from this new beta compiler.

Step 3 above should involve very few changes as `master` was previously
bootstrapping from `beta` which is the same as `stable` at that point in time.
Step 6, however, is where we benefit from removing lots of `#[cfg(stage0)]` and
get to use new features. This also shouldn't slow the release too much as steps
1-5 requires little work other than waiting and step 6 just needs to happen at
some point during a release cycle, it's not time sensitive.

Closes #29555
Closes #29557
@alexcrichton

This comment has been minimized.

Show comment
Hide comment
Member

alexcrichton commented Apr 19, 2016

@bors: r=brson 02538d4

@bors

This comment has been minimized.

Show comment
Hide comment
@bors

bors Apr 19, 2016

Contributor

⌛️ Testing commit 02538d4 with merge 31ffdf4...

Contributor

bors commented Apr 19, 2016

⌛️ Testing commit 02538d4 with merge 31ffdf4...

bors added a commit that referenced this pull request Apr 19, 2016

Auto merge of #32942 - alexcrichton:bootstrap-from-previous, r=brson
mk: Bootstrap from stable instead of snapshots

This commit removes all infrastructure from the repository for our so-called
snapshots to instead bootstrap the compiler from stable releases. Bootstrapping
from a previously stable release is a long-desired feature of distros because
they're not fans of downloading binary stage0 blobs from us. Additionally, this
makes our own CI easier as we can decommission all of the snapshot builders and
start having a regular cadence to when we update the stage0 compiler.

A new `src/etc/get-stage0.py` script was added which shares some code with
`src/bootstrap/bootstrap.py` to read a new file, `src/stage0.txt`, which lists
the current stage0 compiler as well as cargo that we bootstrap from. This script
will download the relevant `rustc` package an unpack it into `$target/stage0` as
we do today.

One problem of bootstrapping from stable releases is that we're not able to
compile unstable code (e.g. all the `#![feature]` directives in libcore/libstd).
To overcome this we employ two strategies:

* The bootstrap key of the previous compiler is hardcoded into `src/stage0.txt`
  (enabled as a result of #32731) and exported by the build system. This enables
  nightly features in the compiler we download.
* The standard library and compiler are pinned to a specific stage0, which
  doesn't change, so we're guaranteed that we'll continue compiling as we start
  from a known fixed source.

The process for making a release will also need to be tweaked now to continue to
cadence of bootstrapping from the previous release. This process looks like:

1. Merge `beta` to `stable`
2. Produce a new stable compiler.
3. Change `master` to bootstrap from this new stable compiler.
4. Merge `master` to `beta`
5. Produce a new beta compiler
6. Change `master` to bootstrap from this new beta compiler.

Step 3 above should involve very few changes as `master` was previously
bootstrapping from `beta` which is the same as `stable` at that point in time.
Step 6, however, is where we benefit from removing lots of `#[cfg(stage0)]` and
get to use new features. This also shouldn't slow the release too much as steps
1-5 requires little work other than waiting and step 6 just needs to happen at
some point during a release cycle, it's not time sensitive.

Closes #29555
Closes #29557
@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Apr 19, 2016

Member

@bors: retry force

Member

alexcrichton commented Apr 19, 2016

@bors: retry force

@bors

This comment has been minimized.

Show comment
Hide comment
@bors

bors Apr 19, 2016

Contributor

⌛️ Testing commit 02538d4 with merge cc0dada...

Contributor

bors commented Apr 19, 2016

⌛️ Testing commit 02538d4 with merge cc0dada...

bors added a commit that referenced this pull request Apr 19, 2016

Auto merge of #32942 - alexcrichton:bootstrap-from-previous, r=brson
mk: Bootstrap from stable instead of snapshots

This commit removes all infrastructure from the repository for our so-called
snapshots to instead bootstrap the compiler from stable releases. Bootstrapping
from a previously stable release is a long-desired feature of distros because
they're not fans of downloading binary stage0 blobs from us. Additionally, this
makes our own CI easier as we can decommission all of the snapshot builders and
start having a regular cadence to when we update the stage0 compiler.

A new `src/etc/get-stage0.py` script was added which shares some code with
`src/bootstrap/bootstrap.py` to read a new file, `src/stage0.txt`, which lists
the current stage0 compiler as well as cargo that we bootstrap from. This script
will download the relevant `rustc` package an unpack it into `$target/stage0` as
we do today.

One problem of bootstrapping from stable releases is that we're not able to
compile unstable code (e.g. all the `#![feature]` directives in libcore/libstd).
To overcome this we employ two strategies:

* The bootstrap key of the previous compiler is hardcoded into `src/stage0.txt`
  (enabled as a result of #32731) and exported by the build system. This enables
  nightly features in the compiler we download.
* The standard library and compiler are pinned to a specific stage0, which
  doesn't change, so we're guaranteed that we'll continue compiling as we start
  from a known fixed source.

The process for making a release will also need to be tweaked now to continue to
cadence of bootstrapping from the previous release. This process looks like:

1. Merge `beta` to `stable`
2. Produce a new stable compiler.
3. Change `master` to bootstrap from this new stable compiler.
4. Merge `master` to `beta`
5. Produce a new beta compiler
6. Change `master` to bootstrap from this new beta compiler.

Step 3 above should involve very few changes as `master` was previously
bootstrapping from `beta` which is the same as `stable` at that point in time.
Step 6, however, is where we benefit from removing lots of `#[cfg(stage0)]` and
get to use new features. This also shouldn't slow the release too much as steps
1-5 requires little work other than waiting and step 6 just needs to happen at
some point during a release cycle, it's not time sensitive.

Closes #29555
Closes #29557
@bors

This comment has been minimized.

Show comment
Hide comment
@bors

bors Apr 19, 2016

Contributor

💔 Test failed - auto-win-msvc-32-opt

Contributor

bors commented Apr 19, 2016

💔 Test failed - auto-win-msvc-32-opt

@alexcrichton

This comment has been minimized.

Show comment
Hide comment
@alexcrichton

alexcrichton Apr 20, 2016

Member

@bors: retry

On Tuesday, April 19, 2016, bors notifications@github.com wrote:

[image: 💔] Test failed - auto-win-msvc-32-opt
http://buildbot.rust-lang.org/builders/auto-win-msvc-32-opt/builds/3067


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#32942 (comment)

Member

alexcrichton commented Apr 20, 2016

@bors: retry

On Tuesday, April 19, 2016, bors notifications@github.com wrote:

[image: 💔] Test failed - auto-win-msvc-32-opt
http://buildbot.rust-lang.org/builders/auto-win-msvc-32-opt/builds/3067


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#32942 (comment)

@bors

This comment has been minimized.

Show comment
Hide comment
@bors

bors Apr 20, 2016

Contributor

⌛️ Testing commit 02538d4 with merge 9bba290...

Contributor

bors commented Apr 20, 2016

⌛️ Testing commit 02538d4 with merge 9bba290...

bors added a commit that referenced this pull request Apr 20, 2016

Auto merge of #32942 - alexcrichton:bootstrap-from-previous, r=brson
mk: Bootstrap from stable instead of snapshots

This commit removes all infrastructure from the repository for our so-called
snapshots to instead bootstrap the compiler from stable releases. Bootstrapping
from a previously stable release is a long-desired feature of distros because
they're not fans of downloading binary stage0 blobs from us. Additionally, this
makes our own CI easier as we can decommission all of the snapshot builders and
start having a regular cadence to when we update the stage0 compiler.

A new `src/etc/get-stage0.py` script was added which shares some code with
`src/bootstrap/bootstrap.py` to read a new file, `src/stage0.txt`, which lists
the current stage0 compiler as well as cargo that we bootstrap from. This script
will download the relevant `rustc` package an unpack it into `$target/stage0` as
we do today.

One problem of bootstrapping from stable releases is that we're not able to
compile unstable code (e.g. all the `#![feature]` directives in libcore/libstd).
To overcome this we employ two strategies:

* The bootstrap key of the previous compiler is hardcoded into `src/stage0.txt`
  (enabled as a result of #32731) and exported by the build system. This enables
  nightly features in the compiler we download.
* The standard library and compiler are pinned to a specific stage0, which
  doesn't change, so we're guaranteed that we'll continue compiling as we start
  from a known fixed source.

The process for making a release will also need to be tweaked now to continue to
cadence of bootstrapping from the previous release. This process looks like:

1. Merge `beta` to `stable`
2. Produce a new stable compiler.
3. Change `master` to bootstrap from this new stable compiler.
4. Merge `master` to `beta`
5. Produce a new beta compiler
6. Change `master` to bootstrap from this new beta compiler.

Step 3 above should involve very few changes as `master` was previously
bootstrapping from `beta` which is the same as `stable` at that point in time.
Step 6, however, is where we benefit from removing lots of `#[cfg(stage0)]` and
get to use new features. This also shouldn't slow the release too much as steps
1-5 requires little work other than waiting and step 6 just needs to happen at
some point during a release cycle, it's not time sensitive.

Closes #29555
Closes #29557
@bors

@bors bors merged commit 02538d4 into rust-lang:master Apr 20, 2016

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
homu Test successful
Details

@alexcrichton alexcrichton deleted the alexcrichton:bootstrap-from-previous branch May 2, 2016

@sanxiyn sanxiyn referenced this pull request Jun 3, 2016

Merged

Unsupport wget #34062

eddyb added a commit to eddyb/rust that referenced this pull request Jun 5, 2016

Rollup merge of #34062 - sanxiyn:no-wget, r=nagisa
Unsupport wget

wget support was removed in #32942 (search for wget in diff), but configure wasn't updated. wget support was introduced in #7498 for Windows, but we now use PowerShell on Windows.

eddyb added a commit to eddyb/rust that referenced this pull request Jun 5, 2016

Rollup merge of #34062 - sanxiyn:no-wget, r=nagisa
Unsupport wget

wget support was removed in #32942 (search for wget in diff), but configure wasn't updated. wget support was introduced in #7498 for Windows, but we now use PowerShell on Windows.

eddyb added a commit to eddyb/rust that referenced this pull request Jun 6, 2016

Rollup merge of #34062 - sanxiyn:no-wget, r=nagisa
Unsupport wget

wget support was removed in #32942 (search for wget in diff), but configure wasn't updated. wget support was introduced in #7498 for Windows, but we now use PowerShell on Windows.

@cuviper cuviper referenced this pull request Jul 1, 2016

Merged

Release notes for 1.10.0 #34591

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment