[thrift] Cannot invoke thrift.exe when cross-compiling on x64 for arm64

[petrmanek]

Describe the bug
I have installed thrift for both triplets x64-windows and arm64-windows. When I attempt to compile my program for x64, everything works as expected. However, when cross-compiling for arm64 I run into an error (see log below).

This happens when thrift IDL is supposed to be translated to C++ stubs and skeletons. My understanding is that the arm64-windows version of the thrift package also ships thrift.exe that was compiled for arm64, and is hence not executable on x64 hosts.

Environment

• OS: Windows 11, latest version as of writing
• Compiler: MSVC 19.38.33134.0

To Reproduce
Steps to reproduce the behavior:

1. ./vcpkg install thrift --triplet x64-windows
2. ./vcpkg install thrift --triplet arm64-windows
3. Build a cmake project with -A x64 that uses thrift.exe, this succeeds
4. Build the same project with -A arm64 that uses thrift.exe, this fails with the error above
   Repro code when

Expected behavior
I would expect that since it generates code (not binaries), thrift.exe is selected based on the host architecture rather than target architecture.

Failure logs

Generating thrift/remote_control_types.h, thrift/remote_control_types.cpp, thrift/RemoteControl.h, thrift/RemoteControl.cpp
  This version of C:\vcpkg\installed\arm64-windows\tools\thrift\thrift.exe is not compatible with the version of Windows you're running. Check your computer's system information and then contact the software publisher.
C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Microsoft\VC\v170\Microsoft.CppCommon.targets(254,5): error MSB8066: Custom build for 'C:\app\build\CMakeFiles\e276fcf5e4251dde3f77451e836d86c9\remote_control_types.h.rule;C:\app\build\CMakeFiles\66f399a733a2e86a9df9ca8604df1574\thriftgen_autogen.rule;C:\app\plugins\remote_control\CMakeLists.txt' exited with code 216. [C:\app\build\plugins\remote_control\thriftgen_autogen.vcxproj]

Additional context
My cmake project finds thrift.exe using the following code.

  find_package(Thrift CONFIG REQUIRED)
  set(Thrift_INCLUDE_DIRS ${THRIFT_INCLUDE_DIR})
  set(Thrift_EXECUTABLE "${THRIFT_BIN_DIR}/thrift.exe")
  set(Thrift_LIBRARIES ${THRIFT_LIBRARIES})

Later IDL is compiled using the following command:

add_custom_command(
  DEPENDS ${CMAKE_CURRENT_SOURCE_DIR}/remote_control.thrift
  OUTPUT
      ${CMAKE_CURRENT_BINARY_DIR}/thrift/remote_control_types.h
      ${CMAKE_CURRENT_BINARY_DIR}/thrift/remote_control_types.cpp
      ${CMAKE_CURRENT_BINARY_DIR}/thrift/RemoteControl.h
      ${CMAKE_CURRENT_BINARY_DIR}/thrift/RemoteControl.cpp
  COMMAND ${CMAKE_COMMAND} -E make_directory ${CMAKE_CURRENT_BINARY_DIR}/thrift
  COMMAND ${Thrift_EXECUTABLE}
  ARGS -gen cpp -r -out ${CMAKE_CURRENT_BINARY_DIR}/thrift ${CMAKE_CURRENT_SOURCE_DIR}/remote_control.thrift)

[FrankXie05]

@petrmanek You should select the corresponding triplet of tools.

[petrmanek]

@petrmanek You should select the corresponding triplet of tools.

What would that be? If I am on x64 and cross-compiling for arm64, I run vcvarsall.bat with amd64_arm64 as the first argument. For vcpkg, I think I am selecting the correct triplet by setting VCPKG_TARGET_ARCHITECTURE=arm64 at cmake configuration stage. Is there anything else I am missing?

[FrankXie05]

@petrmanek The generation of tools thrift.exe is different in different triplets.

[dg0yt]

You should select the corresponding triplet of tools.

The generation of tools thrift.exe is different in different triplets.

I don't see how these comments would help.

find_package(Thrift CONFIG REQUIRED)
...
set(Thrift_EXECUTABLE "${THRIFT_BIN_DIR}/thrift.exe")

This will find Thrift for the target, but it probably won't give you the Thrift_EXECUTABLE to run on the host: I don't see any related corrections in the port, and it also has no dependency on the host thrift.

[petrmanek]

@petrmanek The generation of tools thrift.exe is different in different triplets.

I am aware of this. My issue is that Thrift ships an IDL compiler as well as runtime libraries required by stubs and skeletons generated by said compiler. When cross-compiling, my triplet selection correctly gets me runtime libraries compatible with the right (target) architecture. But unfortunately it also selects IDL compiler compatible with the target architecture, not the host ... hence my issue. To rephrase my question: is there a way to configure vcpkg (via environment, triplet or otherwise) to get me Thrift compiler for the host architecture and Thrift runtime libraries for the target architecture?

I suppose this would be a common problem with Thrift (or any similar middleware for that matter) because its IDL compiler is rarely executed by the end user, as far as I understand it does not even get installed when a program depends on the package. This is understandable as the main use case for it is to generate code at build time, and it stands to reason that the architecture of the compiler binary should be determined by the host, not the target (as it is now).

I don't see how these comments would help.

The set(Thrift_EXECUTABLE "${THRIFT_BIN_DIR}/thrift.exe") is my "invention" and I am aware that it is locking me into the wrong compiler binary. Unfortunately I have not found a better way to locate it programmatically, short of defining a system-wide environment variable and manually pointing it inside a vcpkg package. This is always an option, but I would like to avoid doing it if possible, as it will likely introduce problems down the line whenever the package is updated.

[dg0yt]

You need at least two things:

• Install thrift also for the host triplet
• Set VCPKG_HOST_TRIPLET in your project.

Then the host tool should be available to find_program (via CMAKE_PROGRAM_PATH).

[petrmanek]

Install thrift also for the host triplet

I have done that.

Set VCPKG_HOST_TRIPLET in your project.

I was not aware of this -- will try and report back!

[FrankXie05]

We hope your question was answered to your satisfaction; if it wasn't, you can ping me with more info and I will reopen.

[mschollerer]

Hello together, I like your explination and from a point of view it makes sense. But I think it is not completely solved.

In our multiplattform project we are build for win-64 (native), linux-64 (native) and linux-arm64 (on linux-64 host) using the vcpkg manifest mode (so not being able to add plattform-dependent packages on a certain build, afaik).

So how should I install the thrift-compiler for arm64 build on the host? I think this is not possible with our setup for covering all these targets.

From my point of view the port should recognize if target and host are different and also install the package for the host. This should be its task to enable a successful compilation. I think this is also done for other technologies, like CMake is also installed for the host on cross compile automatically.

Does this sound senseful to you?

Thank you & best regards

[FrankXie05]

So how should I install the thrift-compiler for arm64 build on the host? I think this is not possible with our setup for covering all these targets.

You maybe need install the corresponding thrift:linux-arm64.

[mschollerer]

We install the thrift:linux-arm64, but then the thrift compiler for the host is missing, which needs to be installed to be able to compile the thrift files for arm64. So I would need to install thrift:linux64 additionally as stated here: #36149 (comment), which is not possible through manifest file, as its plattform independent in our project.

[dg0yt]

The manifest can request dependencies with "host": true.

[mschollerer]

That works like a charm, thank you very much!

Incase others are wondering how to retrieve the thrift compiler path for host afterwards, here is our working snippet defining a new variable THRIFT_COMPILER_HOST:
FIND_PROGRAM(THRIFT_COMPILER_HOST thrift PATHS ${CMAKE_PROGRAM_PATH} NO_DEFAULT_PATH REQUIRED)