Enable build-dir layout v2 on nightly by default

[ranger-ross]

What does this PR try to resolve?

This PR enables the new build-dir on nightly by default.

The majority of changes were taken/adapted from #16807

Tracked in #15010

How to test and review this PR?

Same as the original stabilization PR.
The notable changes are:

• A handful of tests may recompile crates when test's p.cargo() use a mixture of stable and nightly
  • For checksum freshness specifically, I enabled nightly on all of the p.cargo() as the tests are specifically testing for rebuilds.

[rustbot]

r? @epage

rustbot has assigned @epage.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Why was this reviewer chosen?

The reviewer was selected based on:

• Owners of files modified in this PR: @epage, @weihanglo
• @epage, @weihanglo expanded to epage, weihanglo
• Random selection from epage, weihanglo

[ranger-ross]

Marking this as ready to review.

I spent some time trying to reproduce sccache failures with argfiles on windows, and couldn't get a failure. (tested on zed again, with __CARGO_TEST_FORCE_ARGFILE but it just worked)
I still wanna tinker on it some more, but I think it should at least be okay on the happy path which makes me more comfortable at least trying it on nightly.

[epage]

Is there a way to do this without removing Default? This makes it more invasive to undo and redo.

View changes since the review

[ranger-ross]

Ehh its get awkward to avoid, specially due to #[serde(default)].

I split out a refactor commit and manually impl'd Default in the macro to reduce the bloat and need to update the Default impl when adding new features.
It's still not ideal but maybe a bit better?

[rustbot]

This PR was rebased onto a different master commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

[weihanglo]

Josh (or someone I don't remember) has mentioned there are people using stable toolchains but enable RUSTC_BOOTSTRAP=1 for enabling some unstable features. Should we avoid enable it if the toolchain is stable/beta?

View changes since the review

[ranger-ross]

It was @Mark-Simulacrum that mentioned it here.
Yeah, I think that makes sense. When that comment was made it kind of went over my head (we have so many bespoke config options 😆)

If I understand it correctly, enabling RUSTC_BOOTSTRAP=1 will result in Cargo using the dev channel.

cargo/src/workspace/features.rs

Lines 1602 to 1605 in 5b6fca7

	if let Ok(staging) = env::var("RUSTC_BOOTSTRAP") {
	if staging == "1" {
	return "dev".to_string();
	}

So the logic should be updated to

let is_rustc_bootstrap = env::var("RUSTC_BOOTSTRAP").map(|o| &o == "1");
unstable.build_dir_new_layout = (c == "nightly" || c == "dev") && !is_new_build_dir_layout_opt_out() && !is_rustc_bootstrap;

Will this cause an issues in bootstrap? I guess not since bootstrap explicitly passes -Zbuild-dir-new-layout and this is just for the defaults.

Does that all sound correct?

[weihanglo]

Either that or just something like crate::version().release_channel == Some("nightly" | "dev")?

[ranger-ross]

ah yeah that should work. I wish I had thought of this before updating all of the tests again. 🫠

Since this skips __CARGO_TEST_CHANNEL_OVERRIDE_DO_NOT_USE_THIS none of the tests need to be updated

[weihanglo]

Thanks

View changes since this review