Running ./x.py test miri --stage 0 twice rebuilds miri, cargo-miri, and rustdoc

[RalfJung]

This is fairly easy to reproduce:

./x.py test miri --stage 0 --test-args hello
./x.py test miri --stage 0 --test-args hello

(The --test-args just serves to make this take less long.)

The second command shouldn't have to build anything, just run the tests. But somehow it actually rebuilds miri, cargo-miri, and rustdoc.

This is a recent regression introduced by #123028. Somehow building rustdoc seems to destroy the build cache for miri and cargo-miri, and vice versa.
Cc @onur-ozkan

[onur-ozkan]

Building rustdoc using the stage 1 compiler causes this, which is weird. I suspect that previously invoked miri related steps may have broken the cache somehow. Or we may need to improve/enhance Builder::ensure.

[RalfJung]

I've just found this in the rustdoc sources:

rust/src/bootstrap/src/core/build_steps/tool.rs

Lines 441 to 445 in 7bdd63d

	// Similar to `compile::Assemble`, build with the previous stage's compiler. Otherwise
	// we'd have stageN/bin/rustc and stageN/bin/rustdoc be effectively different stage
	// compilers, which isn't what we want. Rustdoc should be linked in the same way as the
	// rustc compiler it's paired with, so it must be built with the previous stage compiler.
	let build_compiler = builder.compiler(target_compiler.stage - 1, builder.config.build);

Maybe Miri should do something similar. Then ./x.py test miri should also Do The Right Thing without adding --stage 0 (since actually stage 1 would be the right stage to use). And maybe then the rebuilding issue also vanishes. Or maybe that's unrelated, not sure.

[onur-ozkan]

That would work actually, it seems to be the right path.

[RalfJung]

Unfortunately it doesn't help. We probably still should do it.

But somehow building rustdoc seems to delete the build cache for Miri and cargo-miri, and vice versa.

[RalfJung]

Cargo has this to say about why things are rebuilt:

       Dirty libc v0.2.153: the rustflags changed

And indeed, rustdoc sets

cargo.rustflag("--cfg=parallel_compiler");

That's a bit annoying to have this set in the tool, it means each tool needs to repeat the same logic.

[onur-ozkan]

iirc rustdoc sets that to not dirt the compiler builds, we should probably handle that more globally.

[RalfJung]

iirc rustdoc sets that to not dirt the compiler builds, we should probably handle that more globally.

Yup, that's what the last commit in #123192 does now.