Tool paths on Windows

[mwichmann]

An ongoing problem is finding tool binaries on Windows. And sometimes even making use of them after they're found. Because:

1. SCons considers the standard Windows path for the execution environment (env['ENV']['PATH']) to be very limited: C:/WINDOWS/system32. It does not import os.environ['PATH'] quite intentionally (although users can do so). It's so fundamental it's considerer part of the SCons design.
2. Most software packages for Windows do not install into the "default" path, but into their own directory - some of those add themselves to PATH, some do not. Some install into relatively polite locations like %LOCALAPPDATA% or a Programs subdirectory thereof, for a user-level install, and a subdir of /Program Files for a system-level install. Some do not. And so on.
3. Some software provides a tool to find the actual executables - Visual Studio / MSVC provides vswhere. Python provides the Python launcher py (except that, unhelpfully, the Microsoft Store packaging of Python does not).
4. Some tool modules have built-in ideas of where to find their executable. Java has tables of glob patterns, and uses those; clang/clangxx has a table of Windows paths, and so on. These tools usually add a discovered path to the execution environment.
5. Some tools instead use a promiscuous search (SCons.Util.find_program_path or WhereIs). Whether they then add that path after finding it is not always consistent.
6. For tools available through a package manager like Chocolatey, wrappers to executables may get put in a path standard to that manager. Choco does not do this if the tool seems to be well packaged already (that is, the choco "installer" is just a wrapper around an existing MSI or other installer); for tools which come in a bundle (zip, etc.) they do. SCons adds the Choco path for some of the tool modules, but knows nothing about Scoop, Yarn, winget or various other Windows "package managers".
7. The story has to be repeated for e2e tests, which are standalone programs and don't use SCons data until they actually kick off SCons to run their test code.
8. Predefined paths aren't terribly accurate - most "installers" have an option to put somewhere other than the default; some have changed defaults over time; some offer a choice of "install for everyone, or for just me" which changes where the files go, etc.
9. A user who knows where their executable is can't easily tell SCons to get it from there. Environment has a kwarg toolpath but that's for finding tool modules, not for finding the executables that back them up. An SConscript can create an environment, and then later write to its ['ENV']['PATH'] but if the tool module does extensive initialization based on path to tools, that might be too late. The arguments to Environment are of the overwriting variety - you don't necessarily want to override completely what would have gone into ENV by default. And, of course, if the project is going to be shared, you don't want to put specific paths into the build configuration, as they may not work for other users on different systems.

This whole set of factors creates a scenario that feels very ad-hoc - it's hard to feel there's a consistent picture that tool authors can follow, that test developers can make sure works, etc.

---

A separate topic, because it doesn't have to do with finding programs, is that some external tools require extensive setup/configuration by way of (system) environment variables, which SCons does not pass on, so SCons tool modules for complex compiler suites like the Microsoft and Intel ones need very detailed tool modules to synthesize a working setup that mimics the essential pieces that would have been set up by running the toolchain's setup script in a shell. This is a maintenance headache - it seems almost every new version of Visual Studio brings the need to rework large chunks of the msvc tool, and the intelc tool has rotted to the point that it hasn't worked for several releases now.

the right thing, especially across the evolution of major revisions to those suites. This is a huge burden on the SCons team to keep maintained.