feat: new arg --cargo-config for passing --config to cargo

[weihanglo]

This is another step toward making opt-dist work in sandboxed environments.

--cargo-config is added to the following subcommands:

• binary_stats compile
• bench_runtime_local
• bench_local
• profile_local

Rationale

The current rustc-perf uses tempfile::Tempdir as the working
directory when collecting profiles from some of these packages.
This "tmp" working directory usage make it impossible for Cargo to pick
up the correct vendor sources setting in .cargo/config.toml bundled
in the rustc-src tarball.

We can leverage cargo --config to specify additional Cargo's configuration.
Training data then could build with correct vendored settings via the --cargo-config flag.

See also

• bootstrap: vendor crates required by opt-dist to collect profiles  rust#125465.
• https://rust-lang.zulipchat.com/#narrow/stream/247081-t-compiler.2Fperformance/topic/tempfile.20in.20rustc-perf.20make.20it.20hard.20to.20configure.20vendor

---

r? @Kobzol

(Sorry I haven't figure out how to add tests for this)

[weihanglo]

To integrate this with opt-dist, we need the following patch:

diff --git a/src/tools/opt-dist/src/training.rs b/src/tools/opt-dist/src/training.rs
index 67c850b196d..9e417b01573 100644
--- a/src/tools/opt-dist/src/training.rs
+++ b/src/tools/opt-dist/src/training.rs
@@ -16,7 +16,7 @@ fn init_compiler_benchmarks(
     // Run rustc-perf benchmarks
     // Benchmark using profile_local with eprintln, which essentially just means
     // don't actually benchmark -- just make sure we run rustc a bunch of times.
-    cmd(&[
+    let mut cmd = cmd(&[
         env.cargo_stage_0().as_str(),
         "run",
         "-p",
@@ -41,7 +41,16 @@ fn init_compiler_benchmarks(
     .env("RUST_LOG", "collector=debug")
     .env("RUSTC", env.rustc_stage_0().as_str())
     .env("RUSTC_BOOTSTRAP", "1")
-    .workdir(&env.rustc_perf_dir())
+    .workdir(&env.rustc_perf_dir());
+
+    // Respect `.cargo/config.toml` in the rustc source. This is useful when the
+    // source is from a tarball and contains vendored source settings.
+    let dot_cargo_config_toml  = env.checkout_path().join(".cargo").join("config.toml");
+    if dot_cargo_config_toml.is_file() {
+        cmd.arg("--cargo-config").arg(dot_cargo_config_toml);
+    }
+
+    cmd
 }
 
 /// Describes which `llvm-profdata` binary should be used for merging PGO profiles.

This approach has a strong assumption: if there is a .cargo/config.toml in the root of the rustc source, it will be applied unconditionally.

We could consider other alternatives, such as providing a --cargo-config flag in opt-dist, allowing downstream packagers to use it as needed.

[Kobzol]

Left a few nits, otherwise LGTM.

Could you please also document this in collector/README.md? Thanks!

[weihanglo]

Everything has been pluralized (except the CLI argument name) and simplified. Thank you for your prompt review!

[Kobzol]

Thanks! We'll need to update the rustc-perf version in rust-lang/rust after this.

[weihanglo]

Thanks! We'll need to update the rustc-perf version in rust-lang/rust after this.

Will do that. Maybe after rust-lang/measureme#229?

[Kobzol]

Sure, there are probably more changes in general that need to be made before the builds can be truly self-contained (licensing etc., we'll need to update the rust_team_data dependency).