Non-default cargo locations prefer binaries from default locations over their own

[schultetwin1]

Problem

When I have cargo installed in two places on my machine, both in a non-default location and in the default location, calls to /path/to/nondefault/rust/bin/cargo build will result in ~/.cargo/bin/rustc being used instead of /path/to/nondefault/rust/bin/rustc.

Proposed Solution

In GlobalContext::get_tool() we should add a search path so the full search order is:

    /// 1. From an environment variable matching the tool name (such as `RUSTC`).
    /// 2. From the given config value (which is usually something like `build.rustc`).
    /// 3. From the parent directory of cargo_exe() + tool_str (such as "/path/to/cargo/bin/rustc") <----- THIS ONE IS NEW
    /// 4. Finds the tool in the PATH environment variable.

Notes

This was already somewhat discussed in #1723 in 2015. This is a follow-up to see if others would be open to this idea in 2024.

[weihanglo]

Is there any specific reason that you cannot set RUSTC or other config for your custom tools? If you have your own custom toolchain, rustup can also do that: https://rust-lang.github.io/rustup/concepts/toolchains.html#custom-toolchains

For me, I would avoid adding more complexity to the tool discovery, given there are already ways to do it.

[schultetwin1]

Is there any specific reason that you cannot set RUSTC or other config for your custom tools? If you have your own custom toolchain, rustup can also do that: https://rust-lang.github.io/rustup/concepts/toolchains.html#custom-toolchains

No, there is no (good) specific reasons. I filed this issue is because I think the behavior of cargo here is unexpected. It is nicely documented but we have developers who didn't setup the toolchain, but instead are using it.

These developers come from a C world. If they call /path/to/nondefault/gcc/bin/gcc, they expect gcc to use /path/to/nondefault/gcc/bin/ld and not the version of ld in their path. And that is how that works.

If I have a custom toolchain of Rust on version 1.77, but my default toolchain, install by rustup and in my path is version 1.50, then my intiution is that path/to/rust/1.77/bin/cargo would use path/to/rust/1.77/bin/rustc and not ~/.cargo/bin/rustc, which is much older.

For me, I would avoid adding more complexity to the tool discovery, given there are already ways to do it.

Yea, I understand wanting to avoid additional complexity. Up to you and the cargo maintainers to make the call on what you want to do here!

[workingjubilee]

Huh. What you describe sounds surprising, @schultetwin1, and contradictory to gcc's actual documentation on lookup it seems?

-Bprefix
This option specifies where to find the executables, libraries,
include files, and data files of the compiler itself.

The compiler driver program runs one or more of the subprograms cpp, cc1,
as and ld. It tries prefix as a prefix for each program it tries to run, both
with and without ‘machine/version/’ for the corresponding target machine
and compiler version.

For each subprogram to be run, the compiler driver first tries the -B prefix,
if any. If that name is not found, or if -B is not specified, the driver tries
two standard prefixes, /usr/lib/gcc/ and /usr/local/lib/gcc/. If neither
of those results in a file name that is found, the unmodified program name is
searched for using the directories specified in your PATH environment variable.

Perhaps it is just late and I am misreading their documentation. However, I take that to mean that it does in fact also prefer their "standard" or "default locations" if not overridden using an explicit directive.

[weihanglo]

For reference, here is what Jubilee was talking about: https://gcc.gnu.org/onlinedocs/gcc-13.2.0/gcc/Directory-Options.html#index-B.

---

Also here is a good example of how GCC finds exec files.

Any prefixes specified by the user with -B.
The environment variable GCC_EXEC_PREFIX or, if GCC_EXEC_PREFIX is not set and the compiler has not been installed in the configure-time prefix, the location in which the compiler has actually been installed.
The directories specified by the environment variable COMPILER_PATH.
The macro STANDARD_EXEC_PREFIX, if the compiler has been installed in the configured-time prefix.
The location /usr/libexec/gcc/, but only if this is a native compiler.
The location /usr/lib/gcc/, but only if this is a native compiler.
The macro MD_EXEC_PREFIX, if defined, but only if this is a native compiler. 

As we can see that is pretty GCC or toolchain distributor specific, I'll recommend sticking to whatever rustup provides, or creating a custom toolchain override as aforementioned. Going to close this as rustup is already flexible enough. Let us (or rustup team) know if there is something that doesn't quite fit.

[workingjubilee]

I see, so GCC's documentation is out of date, I suppose.

[schultetwin1]

Thank you for the feedback and documentation links @workingjubilee and @weihanglo! Sounds good to me.