C++ cross compilation from Linux to Windows doesn't work

[tjgq]

When defining a C++ toolchain, paths to the various tools (compiler, linker, etc) must be supplied to cc_toolchain_config. These paths are used as-is if absolute, and otherwise assumed to be relative to the package where the toolchain is defined.

However, the logic to determine whether a path is absolute is host-dependent, so a path like C:/foo/bar is considered relative on Linux (because it doesn't start with /) but absolute on Windows. The result is that a Bazel running on Linux will produce an incorrect command line for a C++ action executed on Windows:

action 'Compiling win_from_linux.cc'
  ...
  Command Line: (exec third_party/toolchains/rbe_windows_bazel_5.2.0_vs2019/C:/VS/VC/Tools/MSVC/14.29.30133/bin/HostX64/x64/cl.exe \
    ...
    win_from_linux.cc)

(In this example, the toolchain was defined in the //third_party/toolchains/rbe_windows_bazel_5.2.0_vs2019 package, and the compiler path was supplied as C:/VS/VC/Tools/MSVC/14.29.30133/bin/HostX64/x64/cl.exe.)

The full repro is available at https://github.com/jtattermusch/win-from-linux-rbe-repro (thanks @jtattermusch!). You must use a Bazel built at or after 2f0948b.

I can think of a few solutions:

• Parameterize the PathFragment absolutization logic by execution platform; likely comes a huge blast radius, since all call sites must be audited and the execution platform might have to be plumbed into places where it's not currently accessible.
• Consider paths starting with a drive letter (X:/...) to be absolute even on non-Windows systems; technically a breaking change, since these are valid relative paths on Unix.
• Make it possible to pass "uninterpreted" paths into cc_toolchain_config (i.e., no absolutization is attempted).

[meisterT]

cc @lberki

[lberki]

Is it possible to not audit every call site of the PathFragment absolutizaton logic but to add an optional argument that says "absolutize as if the path fragment was implemented on this platform"?

Your second option is appealing, but hopefully unnecessary (I don't particularly like carving out "special cases" from the set of all legal Unix paths) but I think taking this little shortcut is better than adding more user-visible knobs.

The third option sounds also reasonable, although it'd be weird to provide either an absolute or a relative path AND a bit that says whether the path is absolute or relative.

So if at all possible, I'd prefer going with the "use the exec platform to select the algorithm to determine whether path is absolute" approach (it looks like the decision is made in CcModule.toolFromStarlark(), so at the very last, it'd have to be moved to somewhere within the bowels of the implementation of cc_toolchain?)

If not, I think parsing C:/foo/cl.exe as an absolute path on Unix is an acceptable compromise.

[sluongng]

Thanks @tjgq for the pointer. We are experiencing this on our end as well.

So if at all possible, I'd prefer going with the "use the exec platform to select the algorithm to determine whether path is absolute" approach (it looks like the decision is made in CcModule.toolFromStarlark(), so at the very last, it'd have to be moved to somewhere within the bowels of the implementation of cc_toolchain?)

I'm in favor of this fix direction as well.

Is it safe to assume that the problem is exclusive to CC toolchain? If that's the case, I think we could create a method API in PathFragment with the exec platform param to narrow down the potential blast radius.