rustup 1.25: On Windows, nested cargo invocation with a toolchain specified fails

[nicholasbishop]

Problem

This is a regression in rustup 1.25 on Windows. If a program run via cargo run tries to run a command like cargo +nightly ..., it fails even if RUSTC is unset in the environment.

Splitting this off from #3031 (comment) because #3031 (comment) indicates it's a separate problem.

Steps

https://github.com/nicholasbishop/rustup-nested-bug-repro#rustup-nested-bug-repro contains a repro of this bug.

Here's the main.rs of that repo:

use std::process::Command;
use std::env;

fn main() {
    // Modify the path if any argument is passed in.
    let modify_path = env::args().len() == 2;

    let mut cmd = Command::new("cargo");
    cmd.env_remove("RUSTC");
    cmd.env_remove("RUSTDOC");
    cmd.args(&["+nightly", "version"]);

    if modify_path {
        let orig_path = env::var_os("PATH").unwrap_or_default();
        let modified_split_paths = env::split_paths(&orig_path).filter(|path| {
            !path
                .components()
                .any(|component| component.as_os_str() == ".rustup")
        });
        let modified_path = env::join_paths(modified_split_paths).expect("invalid PATH");
        cmd.env("PATH", modified_path);
    }

    println!("running command: {:?}", cmd);
    let s = cmd.status().unwrap();
    assert!(s.success());
}

On Linux this command succeeds:

$ cargo run
   Compiling rustup-nested-bug-repro v0.1.0 (/var/home/nbishop/src/rustup-nested-bug-repro)
    Finished dev [unoptimized + debuginfo] target(s) in 0.17s
     Running `target/debug/rustup-nested-bug-repro`
running command: "cargo" "+nightly" "version"
cargo 1.64.0-nightly (b1dd22e66 2022-07-09)

On Windows it fails:

$ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.18s
     Running `target\debug\rustup-nested-bug-repro.exe`
running command: "cargo" "+nightly" "version"
error: no such subcommand: `+nightly`
thread 'main' panicked at 'assertion failed: s.success()', src\main.rs:26:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: process didn't exit successfully: `target\debug\rustup-nested-bug-repro.exe` (exit code: 101)

But it succeeds if the PATH is modified to remove toolchain entries
added by rustup:

$ cargo run modify
    Finished dev [unoptimized + debuginfo] target(s) in 0.17s
     Running `target\debug\rustup-nested-bug-repro.exe modify`
running command: "cargo" "+nightly" "version"
cargo 1.64.0-nightly (b1dd22e66 2022-07-09)

Possible Solution(s)

No response

Notes

No response

Rustup version

$ rustup --version
rustup 1.25.0 (90365aa81 2022-06-15)
info: This is the version for the rustup toolchain manager, not the rustc compiler.
info: The currently active `rustc` version is `rustc 1.62.0 (a8314ef7d 2022-06-27)`

Installed toolchains

$ rustup show
Default host: x86_64-pc-windows-msvc
rustup home:  C:\Users\nicholasbishop\.rustup

installed toolchains
--------------------

stable-x86_64-pc-windows-msvc (default)
nightly-x86_64-pc-windows-msvc

active toolchain
----------------

stable-x86_64-pc-windows-msvc (default)
rustc 1.62.0 (a8314ef7d 2022-06-27)

[jyn514]

@nicholasbishop can you change your program to print RUSTUP_TOOLCHAIN, PATH, and the arguments, and run it under both 1.24.1 and 1.25.0 so we can see the difference?

[nicholasbishop]

Sure thing. What's the recommended way to install an older version of rustup?

[jyn514]

Hmm, I'm not sure we keep around older versions - you may need to build from source: https://github.com/rust-lang/rustup/releases/tag/1.24.3

[pietroalbini]

@nicholasbishop you can download the installer from either URLs, depending on the extension of the OS:

• https://static.rust-lang.org/rustup/archive/${VERSION}/${TARGET}/rustup-init
• https://static.rust-lang.org/rustup/archive/${VERSION}/${TARGET}/rustup-init.exe

So, for example:

• https://static.rust-lang.org/rustup/archive/1.24.1/x86_64-unknown-linux-gnu/rustup-init
• https://static.rust-lang.org/rustup/archive/1.24.1/x86_64-pc-windows-msvc/rustup-init.exe

[nicholasbishop]

Thanks. Here's what it looks like in 1.24.3:

$ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.01s
     Running `target\debug\rustup-nested-bug-repro.exe`
RUSTUP_TOOLCHAIN: "stable-x86_64-pc-windows-msvc"
Original PATH: "C:\\Users\\nicholasbishop\\src\\rustup-nested-bug-repro\\target\\debug\\deps;C:\\Users\\nicholasbishop\\src\\rustup-nested-bug-repro\\target\\debug;C:\\Users\\nicholasbishop\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib;C:\\Users\\nicholasbishop\\.cargo\\bin;C:\\Users\\nicholasbishop\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\bin;C:\\Users\\nicholasbishop\\AppData\\Local\\Programs\\Python\\Python310\\;C:\\Users\\nicholasbishop\\bin;C:\\Program Files\\Git\\mingw64\\bin;C:\\Program Files\\Git\\usr\\local\\bin;C:\\Program Files\\Git\\usr\\bin;C:\\Program Files\\Git\\usr\\bin;C:\\Program Files\\Git\\mingw64\\bin;C:\\Program Files\\Git\\usr\\bin;C:\\Users\\nicholasbishop\\bin;C:\\Program Files\\Oculus\\Support\\oculus-runtime;C:\\Program Files (x86)\\Razer Chroma SDK\\bin;C:\\Program Files\\Razer Chroma SDK\\bin;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0;C:\\WINDOWS\\System32\\OpenSSH;C:\\Program Files\\Git\\cmd;C:\\Users\\nicholasbishop\\.cargo\\bin;C:\\Users\\nicholasbishop\\AppData\\Local\\Microsoft\\WindowsApps;C:\\Program Files\\Git\\usr\\bin\\vendor_perl;C:\\Program Files\\Git\\usr\\bin\\core_perl"
running command: "cargo" "+nightly" "version"
cargo 1.64.0-nightly (b1dd22e66 2022-07-09)

And in 1.25.0:

$ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.17s
     Running `target\debug\rustup-nested-bug-repro.exe`
RUSTUP_TOOLCHAIN: "stable-x86_64-pc-windows-msvc"
Original PATH: "C:\\Users\\nicholasbishop\\src\\rustup-nested-bug-repro\\target\\debug\\deps;C:\\Users\\nicholasbishop\\src\\rustup-nested-bug-repro\\target\\debug;C:\\Users\\nicholasbishop\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\lib\\rustlib\\x86_64-pc-windows-msvc\\lib;C:\\Users\\nicholasbishop\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\bin;C:\\Users\\nicholasbishop\\AppData\\Local\\Programs\\Python\\Python310\\;C:\\Users\\nicholasbishop\\bin;C:\\Program Files\\Git\\mingw64\\bin;C:\\Program Files\\Git\\usr\\local\\bin;C:\\Program Files\\Git\\usr\\bin;C:\\Program Files\\Git\\usr\\bin;C:\\Program Files\\Git\\mingw64\\bin;C:\\Program Files\\Git\\usr\\bin;C:\\Users\\nicholasbishop\\bin;C:\\Program Files\\Oculus\\Support\\oculus-runtime;C:\\Program Files (x86)\\Razer Chroma SDK\\bin;C:\\Program Files\\Razer Chroma SDK\\bin;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0;C:\\WINDOWS\\System32\\OpenSSH;C:\\Program Files\\Git\\cmd;C:\\Users\\nicholasbishop\\.cargo\\bin;C:\\Users\\nicholasbishop\\AppData\\Local\\Microsoft\\WindowsApps;C:\\Program Files\\Git\\usr\\bin\\vendor_perl;C:\\Program Files\\Git\\usr\\bin\\core_perl"
running command: "cargo" "+nightly" "version"
error: no such subcommand: `+nightly`
thread 'main' panicked at 'assertion failed: s.success()', src\main.rs:32:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: process didn't exit successfully: `target\debug\rustup-nested-bug-repro.exe` (exit code: 101)

Seems like the significant change between them might be the order of the entries. C:\\Users\\nicholasbishop\\.cargo\\bin comes before C:\\Users\\nicholasbishop\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\bin in 1.24.3, but after it in 1.25.0.

[ChrisDenton]

Possibly related to #2978? Not sure. Edit: Hm, seems cargo\bin is listed twice in 1.24.3 but not in 1.25.0. And because 1.25.0 skips prepending it to PATH, it effectively changes the order. @jonhoo

[CAD97]

For future visitors: rustup run toolchain cargo seems to work in the desired manner, at least w.r.t. this issue.

[kinnison]

Is this resolved in 1.25.1 or do we need to still find out what the right fix is?

[lnicola]

Still a problem according to rust-lang/rust-analyzer#12953.

[jonhoo]

Hm, seems cargo\bin is listed twice in 1.24.3 but not in 1.25.0. And because 1.25.0 skips prepending it to PATH, it effectively changes the order.

This was entirely an intentional change. It's not clear that Rustup should change $PATH in the way it used to — it's effectively hoisting the priority of any Rust program the user has installed above the existing priority order of $PATH.

In my mind, the bug here is that C:\\Users\\nicholasbishop\\.rustup\\toolchains\\stable-x86_64-pc-windows-msvc\\bin is on $PATH in the first place and is listed before C:\\Users\\nicholasbishop\\.cargo\\bin. We should track down the origin of that, because that's the real (configuration) issue here. I think such a setup should have been broken, and just worked accidentally because Rustup was making changes to $PATH that it's not supposed to make.

[ehuss]

I did some digging and opened #3178 as a solution for this issue.

[ehuss]

@rbtcollins Can you reopen this since the change is still opt-in?