Improve defaults in x.py #73964

jyn514 · 2020-07-02T13:14:25Z

Make the default stage dependent on the subcommand
Don't build stage1 rustc artifacts with x.py build --stage 1. If this is what you want, use x.py build --stage 2 instead, which gives you a working libstd.
Change default debuginfo when debug = true from 2 to 1

I tried to fix CI to use --stage 2 everywhere it currently has no stage, but I might have missed a spot.
This does not update much of the documentation - most of it is in https://github.com/rust-lang/rustc-dev-guide/ or https://github.com/rust-lang/rust-forge and will need a separate PR.

See individual commits for a detailed rationale of each change.
See also the MCP: rust-lang/compiler-team#326

r? @Mark-Simulacrum , but anyone is free to give an opinion.

jyn514 · 2020-07-02T13:16:28Z

I would also like to add an option to build rustc without libstd at some point. This is not currently possible: build --stage 0 will compile rustc but won't assemble it (#73519) and I think the same is true for --stage 1.

config.toml.example

jyn514 · 2020-07-02T14:10:15Z

Looking at https://forge.rust-lang.org/compiler/mcp.html this should definitely be an MCP so marking as draft until I've gone through that process.

src/bootstrap/builder.rs

petrochenkov · 2020-07-02T17:30:24Z

The current defaults (except for the default stage) minimize the edit-build-test cycle, i.e. serve the use case "build rustc as fast as possible to run tests".

The new defaults are debug-oriented - you have some issue and build the compiler with extra info, spending extra time on producing debuginfo, assertions and logging.

The first scenario is much more common from my experience.
So I use the current default settings most of the time with debuginfo-level = 1 being the only exceptions (it's very cheap and doesn't cause any observable slowdown).

jyn514 · 2020-07-02T18:02:20Z

The current defaults (except for the default stage) minimize the edit-build-test cycle, i.e. serve the use case "build rustc as fast as possible to run tests".

The new defaults are debug-oriented - you have some issue and build the compiler with extra info, spending extra time on producing debuginfo, assertions and logging.

The first scenario is much more common from my experience.
So I use the current default settings most of the time with debuginfo-level = 1 being the only exceptions (it's very cheap and doesn't cause any observable slowdown).

The debug = true change is the one I feel the least strongly about. If you think that should be kept as debug = false by default I'm fine with that as long as there's a suggestion in rustc-dev-guide to switch it on. I will note though that the people with the use case to "build rustc as fast as possible to run tests" are probably experienced contributors and know to look for debug = false in config.toml.

What do you think about the rest of the changes?

CAD97 · 2020-07-02T21:07:32Z

I will note though that the people with the use case to "build rustc as fast as possible to run tests" are probably experienced contributors

I'd argue the exact opposite. People new to hacking on rustc are the most likely to use an unqualified ./x.py test in order to test their changes. And they definitely just want their test results, and don't really care what's done to get to that point.

joshtriplett · 2020-07-03T04:33:12Z

I would personally like to have debug = false by default` as well. I think it makes sense to provide better build performance out of the box; most people won't run rustc through a debugger or similar.

That said, if debug with debug level 1 causes almost no slowdown, and doesn't use substantially more memory when building or linking, then leaving that on by default seems fine.

jyn514 · 2020-07-03T04:40:13Z

To be clear, debug = true does not mean you can run rustc through a debugger, for that you need debuginfo = 2. Instead it adds line numbers to backtraces, enables debug assertions, and emits debug logging. Without debug = true, RUST_LOG=debug has exactly the same effect as RUST_LOG=info, which can be very confusing for new contributors.

joshtriplett · 2020-07-03T05:24:55Z

@jyn514 Ah, I see; that seems reasonable. Thanks for the explanation.

jyn514 · 2020-07-03T13:03:39Z

I removed debug = true from the MCP, see rust-lang/compiler-team#326 (comment) for discussion.

pickfire · 2020-07-03T14:21:14Z

To be clear, debug = true does not mean you can run rustc through a debugger, for that you need debuginfo = 2. Instead it adds line numbers to backtraces, enables debug assertions, and emits debug logging. Without debug = true, RUST_LOG=debug has exactly the same effect as RUST_LOG=info, which can be very confusing for new contributors.

The worst thing I did was changing that, I need to rebuild (which took me a day) just to flip the debug to true.

jyn514 · 2020-07-07T23:34:16Z

The failure is reproducible with

$ ./x.py test --doc src/tools/rustdoc
   Doc-tests rustdoc

running 1 test
F
failures:

---- html/markdown.rs - html::markdown (line 5) stdout ----
Test executable failed (exit code 127).

stderr:
/tmp/rustdoctestmt2Lhu/rust_out: error while loading shared libraries: libtest-762795f83cf44985.so: cannot open shared object file: No such file or directory

Investigating now.

jyn514 · 2020-07-14T00:57:52Z

Yay bootstrap tests are finally passing 🎉 next to change CI to use the explicit names for things instead of relying on the defaults to be correct.

jyn514 · 2020-07-14T03:58:35Z

CI is passing and using stage 2 🎉 The MCP still has 4 days until it's officially accepted, but this is ready for review.

petrochenkov · 2020-07-14T05:53:26Z

@jyn514
Could you split the changes for different defaults into separate PRs? (One for stage 2, one for debuginfo, etc.)
Right now this PR does many different things, moves code around and doesn't maintain git history.
The PR description also went out of sync with its contents.
All of this makes a person who wants to figure out what this PR changes to do a full review.

jyn514 · 2020-07-28T13:39:51Z

@bors r=Mark-Simulacrum rollup=iffy

bors · 2020-07-28T13:39:52Z

📌 Commit da40cf8 has been approved by Mark-Simulacrum

bors · 2020-07-28T13:56:38Z

⌛ Testing commit da40cf8 with merge 7b3a781...

bors · 2020-07-28T17:07:11Z

☀️ Test successful - checks-actions, checks-azure
Approved by: Mark-Simulacrum
Pushing 7b3a781 to master...

See rust-lang/rust#73964. The choices are either to link to `stage1` with rustup or build stage 2; this recommends to build stage2 to be consistent with the existing docs.

matklad · 2020-08-30T09:08:52Z

I’ve just learned about this from 1.47 relnotes. This might mean that this might need some additional advertising for existing contributors to take advantage of. “How rustc stages work” might be an interesting post for inside Rust blog :)

tesuji · 2020-08-30T09:13:01Z

Joshua is making a post: rust-lang/blog.rust-lang.org#666

After rust-lang/rust#73964, they're redundant.

Pkgsrc changes: * Remove patches now integrated upstream, many related to SunOS / Illumos. * The LLVM fix for powerpc is also now integrated upstream. * Adapt those patches where the source has moved or parts are integrated. * The randomness patches no longer applies, and I could not find where those files went... * Provide a separate bootstrap for NetBSD/powerpc 9.0, since apparently the C++ ABI is different from 8.0. Yes, this appears to be specific to the NetBSD powerpc ports. Upstream changes: Version 1.47.0 (2020-10-08) ========================== Language -------- - [Closures will now warn when not used.][74869] Compiler -------- - [Stabilized the `-C control-flow-guard` codegen option][73893], which enables [Control Flow Guard][1.47.0-cfg] for Windows platforms, and is ignored on other platforms. - [Upgraded to LLVM 11.][73526] - [Added tier 3\* support for the `thumbv4t-none-eabi` target.][74419] - [Upgrade the FreeBSD toolchain to version 11.4][75204] - [`RUST_BACKTRACE`'s output is now more compact.][75048] \* Refer to Rust's [platform support page][forge-platform-support] for more information on Rust's tiered platform support. Libraries --------- - [`CStr` now implements `Index<RangeFrom<usize>>`.][74021] - [Traits in `std`/`core` are now implemented for arrays of any length, not just those of length less than 33.][74060] - [`ops::RangeFull` and `ops::Range` now implement Default.][73197] - [`panic::Location` now implements `Copy`, `Clone`, `Eq`, `Hash`, `Ord`, `PartialEq`, and `PartialOrd`.][73583] Stabilized APIs --------------- - [`Ident::new_raw`] - [`Range::is_empty`] - [`RangeInclusive::is_empty`] - [`Result::as_deref`] - [`Result::as_deref_mut`] - [`Vec::leak`] - [`pointer::offset_from`] - [`f32::TAU`] - [`f64::TAU`] The following previously stable APIs have now been made const. - [The `new` method for all `NonZero` integers.][73858] - [The `checked_add`,`checked_sub`,`checked_mul`,`checked_neg`, `checked_shl`, `checked_shr`, `saturating_add`, `saturating_sub`, and `saturating_mul` methods for all integers.][73858] - [The `checked_abs`, `saturating_abs`, `saturating_neg`, and `signum` for all signed integers.][73858] - [The `is_ascii_alphabetic`, `is_ascii_uppercase`, `is_ascii_lowercase`, `is_ascii_alphanumeric`, `is_ascii_digit`, `is_ascii_hexdigit`, `is_ascii_punctuation`, `is_ascii_graphic`, `is_ascii_whitespace`, and `is_ascii_control` methods for `char` and `u8`.][73858] Cargo ----- - [`build-dependencies` are now built with opt-level 0 by default.][cargo/8500] You can override this by setting the following in your `Cargo.toml`. ```toml [profile.release.build-override] opt-level = 3 ``` - [`cargo-help` will now display man pages for commands rather just the `--help` text.][cargo/8456] - [`cargo-metadata` now emits a `test` field indicating if a target has tests enabled.][cargo/8478] - [`workspace.default-members` now respects `workspace.exclude`.][cargo/8485] - [`cargo-publish` will now use an alternative registry by default if it's the only registry specified in `package.publish`.][cargo/8571] Misc ---- - [Added a help button beside Rustdoc's searchbar that explains rustdoc's type based search.][75366] - [Added the Ayu theme to rustdoc.][71237] Compatibility Notes ------------------- - [Bumped the minimum supported Emscripten version to 1.39.20.][75716] - [Fixed a regression parsing `{} && false` in tail expressions.][74650] - [Added changes to how proc-macros are expanded in `macro_rules!` that should help to preserve more span information.][73084] These changes may cause compiliation errors if your macro was unhygenic or didn't correctly handle `Delimiter::None`. - [Moved support for the CloudABI target to tier 3.][75568] - [`linux-gnu` targets now require minimum kernel 2.6.32 and glibc 2.11.][74163] Internal Only -------- - [Improved default settings for bootstrapping in `x.py`.][73964] You can read details about this change in the ["Changes to `x.py` defaults"](https://blog.rust-lang.org/inside-rust/2020/08/30/changes-to-x-py-defaults.html) post on the Inside Rust blog. - [Added the `rustc-docs` component.][75560] This allows you to install and read the documentation for the compiler internal APIs. (Currently only available for `x86_64-unknown-linux-gnu`.) [1.47.0-cfg]: https://docs.microsoft.com/en-us/windows/win32/secbp/control-flow-guard [76980]: rust-lang/rust#76980 [75048]: rust-lang/rust#75048 [74163]: rust-lang/rust#74163 [71237]: rust-lang/rust#71237 [74869]: rust-lang/rust#74869 [73858]: rust-lang/rust#73858 [75716]: rust-lang/rust#75716 [75908]: rust-lang/rust#75908 [75516]: rust-lang/rust#75516 [75560]: rust-lang/rust#75560 [75568]: rust-lang/rust#75568 [75366]: rust-lang/rust#75366 [75204]: rust-lang/rust#75204 [74650]: rust-lang/rust#74650 [74419]: rust-lang/rust#74419 [73964]: rust-lang/rust#73964 [74021]: rust-lang/rust#74021 [74060]: rust-lang/rust#74060 [73893]: rust-lang/rust#73893 [73526]: rust-lang/rust#73526 [73583]: rust-lang/rust#73583 [73084]: rust-lang/rust#73084 [73197]: rust-lang/rust#73197 [72488]: rust-lang/rust#72488 [cargo/8456]: rust-lang/cargo#8456 [cargo/8478]: rust-lang/cargo#8478 [cargo/8485]: rust-lang/cargo#8485 [cargo/8500]: rust-lang/cargo#8500 [cargo/8571]: rust-lang/cargo#8571 [`Ident::new_raw`]: https://doc.rust-lang.org/nightly/proc_macro/struct.Ident.html#method.new_raw [`Range::is_empty`]: https://doc.rust-lang.org/nightly/std/ops/struct.Range.html#method.is_empty [`RangeInclusive::is_empty`]: https://doc.rust-lang.org/nightly/std/ops/struct.RangeInclusive.html#method.is_empty [`Result::as_deref_mut`]: https://doc.rust-lang.org/nightly/std/result/enum.Result.html#method.as_deref_mut [`Result::as_deref`]: https://doc.rust-lang.org/nightly/std/result/enum.Result.html#method.as_deref [`TypeId::of`]: https://doc.rust-lang.org/nightly/std/any/struct.TypeId.html#method.of [`Vec::leak`]: https://doc.rust-lang.org/nightly/std/vec/struct.Vec.html#method.leak [`f32::TAU`]: https://doc.rust-lang.org/nightly/std/f32/consts/constant.TAU.html [`f64::TAU`]: https://doc.rust-lang.org/nightly/std/f64/consts/constant.TAU.html [`pointer::offset_from`]: https://doc.rust-lang.org/nightly/std/primitive.pointer.html#method.offset_from

rust-highfive assigned Mark-Simulacrum Jul 2, 2020

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Jul 2, 2020

teymour-aldridge reviewed Jul 2, 2020

View reviewed changes

config.toml.example Outdated Show resolved Hide resolved

jyn514 marked this pull request as draft July 2, 2020 14:10

jyn514 changed the title ~~Use sane defaults in x.py~~ [Draft] Use sane defaults in x.py Jul 2, 2020

jyn514 mentioned this pull request Jul 2, 2020

Improve defaults in x.py rust-lang/compiler-team#326

Closed

3 tasks

Mark-Simulacrum added S-blocked Status: Marked as blocked ❌ on something else such as an RFC or other implementation work. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 2, 2020

pickfire reviewed Jul 2, 2020

View reviewed changes

src/bootstrap/builder.rs Outdated Show resolved Hide resolved

petrochenkov assigned petrochenkov and unassigned petrochenkov Jul 2, 2020

jyn514 changed the title ~~[Draft] Use sane defaults in x.py~~ [Draft] Improve defaults in x.py Jul 7, 2020

jyn514 mentioned this pull request Jul 11, 2020

internal compiler error: src/librustc_middle/ty/query/mod.rs:105:1: tcx.dep_kind(crate0) unsupported by its crate #74258

Closed

jyn514 force-pushed the sane-defaults branch from bfed930 to 88eb19a Compare July 13, 2020 21:39

jyn514 changed the title ~~[Draft] Improve defaults in x.py~~ Improve defaults in x.py Jul 14, 2020

jyn514 marked this pull request as ready for review July 14, 2020 04:00

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jul 28, 2020

bors added the merged-by-bors This PR was explicitly merged by bors. label Jul 28, 2020

bors merged commit 7b3a781 into rust-lang:master Jul 28, 2020

jyn514 deleted the sane-defaults branch July 28, 2020 17:09

bors mentioned this pull request Jul 28, 2020

Re-enable linkcheck after moving std #74861

Merged

This was referenced Aug 5, 2020

full-bootstrap builder doesn't test anything #75198

Closed

Display elided lifetime for non-reference type in doc #75237

Merged

mati865 mentioned this pull request Aug 11, 2020

Ignore gated-plugin test on stage1 #75404

Closed

jyn514 mentioned this pull request Aug 15, 2020

Add --run-stage and --link-stage aliases #75167

Closed

jyn514 mentioned this pull request Aug 16, 2020

Update README for changes in x.py defaults rust-lang/measureme#124

Merged

jyn514 mentioned this pull request Aug 25, 2020

[WIP] Add rustdoc GUI tests #70533

Closed

jyn514 added a commit to jyn514/rustc-dev-guide that referenced this pull request Aug 31, 2020

Use fewer stages in bootstrapping docs

2662e22

After rust-lang/rust#73964, they're redundant.

jyn514 mentioned this pull request Aug 31, 2020

Use fewer stages in bootstrapping docs rust-lang/rustc-dev-guide#839

Merged

jyn514 added a commit to jyn514/rustc-dev-guide that referenced this pull request Aug 31, 2020

Use fewer stages in bootstrapping docs

d6606d5

After rust-lang/rust#73964, they're redundant.

jyn514 added a commit to jyn514/rustc-dev-guide that referenced this pull request Aug 31, 2020

Use fewer stages in bootstrapping docs

c34cc2c

After rust-lang/rust#73964, they're redundant.

tshepang pushed a commit to rust-lang/rustc-dev-guide that referenced this pull request Aug 31, 2020

Use fewer stages in bootstrapping docs

2924b69

After rust-lang/rust#73964, they're redundant.

This was referenced Aug 31, 2020

Make the default stage for x.py configurable #76165

Closed

Add a stable version of x.py #76380

Closed

jyn514 mentioned this pull request Sep 9, 2020

Add x.py setup or similar #76503

Closed

bjorn3 mentioned this pull request Jan 24, 2021

Many tests failing with "Couldn't compile the test" on 32-bit PowerPC with 1.49 beta #81334

Closed

cuviper added this to the 1.47.0 milestone May 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve defaults in x.py #73964

Improve defaults in x.py #73964

jyn514 commented Jul 2, 2020 •

edited

jyn514 commented Jul 2, 2020 •

edited

jyn514 commented Jul 2, 2020

petrochenkov commented Jul 2, 2020

jyn514 commented Jul 2, 2020

CAD97 commented Jul 2, 2020

joshtriplett commented Jul 3, 2020 •

edited

jyn514 commented Jul 3, 2020 •

edited

joshtriplett commented Jul 3, 2020

jyn514 commented Jul 3, 2020

pickfire commented Jul 3, 2020

jyn514 commented Jul 7, 2020

jyn514 commented Jul 14, 2020

jyn514 commented Jul 14, 2020

petrochenkov commented Jul 14, 2020 •

edited

jyn514 commented Jul 28, 2020

bors commented Jul 28, 2020

bors commented Jul 28, 2020

bors commented Jul 28, 2020

matklad commented Aug 30, 2020

tesuji commented Aug 30, 2020

Improve defaults in x.py #73964

Improve defaults in x.py #73964

Conversation

jyn514 commented Jul 2, 2020 • edited

jyn514 commented Jul 2, 2020 • edited

jyn514 commented Jul 2, 2020

petrochenkov commented Jul 2, 2020

jyn514 commented Jul 2, 2020

CAD97 commented Jul 2, 2020

joshtriplett commented Jul 3, 2020 • edited

jyn514 commented Jul 3, 2020 • edited

joshtriplett commented Jul 3, 2020

jyn514 commented Jul 3, 2020

pickfire commented Jul 3, 2020

jyn514 commented Jul 7, 2020

jyn514 commented Jul 14, 2020

jyn514 commented Jul 14, 2020

petrochenkov commented Jul 14, 2020 • edited

jyn514 commented Jul 28, 2020

bors commented Jul 28, 2020

bors commented Jul 28, 2020

bors commented Jul 28, 2020

matklad commented Aug 30, 2020

tesuji commented Aug 30, 2020

jyn514 commented Jul 2, 2020 •

edited

jyn514 commented Jul 2, 2020 •

edited

joshtriplett commented Jul 3, 2020 •

edited

jyn514 commented Jul 3, 2020 •

edited

petrochenkov commented Jul 14, 2020 •

edited