-
Notifications
You must be signed in to change notification settings - Fork 6.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[protobuf][protoc][grpc] Better support when cross compiling #26621
Conversation
invalid. Use a host dependency |
Are you talking about using the "--host-triplet" option and this document https://github.com/microsoft/vcpkg/blob/master/docs/users/host-dependencies.md? I am executing the following:
with the (maybe weird) intention of building a vcpkg installation directory that can be delivered to an arm64 machine as a pre-built dependency cache (basically There are also references to I understand that arm64 Macs can run x86 binaries with Rosetta, but I am going for correctness here by ensuring correct architecture binaries in their respective directories and CMake configurations that work when faced with the above scenario. I am also now realizing that I don't understand the whole picture of how vcpkg handles cross-compilation. Is there a better way to accomplish what I'm trying to do? |
0a9d6cb
to
f07c54f
Compare
f07c54f
to
ada7387
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You have modified or added at least one vcpkg.json where you should check the license
field.
If you feel able to do so, please consider adding a "license" field to the following files:
ports/cartographer/vcpkg.json
ports/offscale-libetcd-cpp/vcpkg.json
ports/srpc/vcpkg.json
Valid values for the license field can be found in the documentation
ada7387
to
df497a8
Compare
`protoc` tool should always be built and be compatible for the target system. Fix upb finding protoc executable and support cross compilation. grpc remove targets file
df497a8
to
41183f8
Compare
add_custom_command( | ||
OUTPUT ${common_protofiles} | ||
- COMMAND protobuf::protoc | ||
+ COMMAND "${Protobuf_PROTOC_EXECUTABLE}" | ||
ARGS --cpp_out=${generated_path} -I ${protodir_path} ${protos} | ||
- DEPENDS ${protos} protobuf::protoc | ||
+ DEPENDS ${protos} | ||
COMMENT "Running cpp protocol buffer compiler on ${protos}" | ||
VERBATIM ) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can we not simply fix the target protobuf::protoc
for crosscompiling instead of patching everyone downstream?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good question. I'm unsure of the best solution, and I'm still trying to wrap my head around CMake cross compilation best practices and what vcpkg is doing.
I was under the impression that it's probably best to expect that any exported targets from a find_package
call are all the same architecture, OS, etc... However, as I keep investigating all of this, I wonder how useful an exported executable target is if it needs to be replaced or goes unused during cross compilation. It seems like a cross-compilation-compatible solution would either:
- Rely on
find_program
to find all host executables that will be run during the build - Separate config files for library and executable targets where the user would need to either specify an exact
*_DIR
path to the host config file or (after much testing) rely onfind_package
withNO_CMAKE_PATH
,NO_CMAKE_FIND_ROOT_PATH
, andHINTS ${CMAKE_SYSTEM_PREFIX_PATH}
options to search only the host directories,1 i.e.,find_package(Protobuf CONFIG HINTS ${CMAKE_SYSTEM_PREFIX_PATH} NO_CMAKE_PATH NO_CMAKE_FIND_ROOT_PATH)
fix the target protobuf::protoc
I believe what you're describing is attempted with these lines
find_program(Protobuf_PROTOC_EXECUTABLE NAMES protoc PATHS "${CMAKE_CURRENT_LIST_DIR}/../../tools/protobuf" NO_DEFAULT_PATH) |
vcpkg/ports/protobuf/protobuf-targets-vcpkg-protoc.cmake
Lines 4 to 8 in 1eb4c7f
# Import target "protobuf::protoc" for configuration "Release" | |
set_property(TARGET protobuf::protoc APPEND PROPERTY IMPORTED_CONFIGURATIONS RELEASE) | |
set_target_properties(protobuf::protoc PROPERTIES | |
IMPORTED_LOCATION_RELEASE "${Protobuf_PROTOC_EXECUTABLE}" | |
) |
I've removed these files in my PR, but I should investigate this again as you suggest. However, these files, being only in vcpkg, would hide downstream projects' issues if they tried to cross compile outside of vcpkg without also fixing the target. What I mean is that I think there is room for a patch in upstream protobuf for better cross compilation support, and I'm wondering what a good solution is and want to test it here.
list(INSERT CMAKE_SYSTEM_PREFIX_PATH 0 "${VCPKG_INSTALLED_DIR}/${VCPKG_HOST_TRIPLET}") # CMake 3.15 is required for list(PREPEND ...).
Footnotes
-
Vcpkg currently doesn't touch
CMAKE_SYSTEM_PREFIX_PATH
, so we'd want to add the path to the host triplet installation root. I've done very light testing locally by adding the following toscripts/buildsystem/vcpkg.cmake
: ↩
Pinging @ekilmer for response. Is work still being done for this PR? |
@JonLiu1993 I think I need to approach this differently. I haven't had much time to investigate, so I'll close this for now. Maybe I'll reopen this or another PR in the future when I have more time. |
protoc
tool should always be built and be compatible for the targetsystem.
Fix
upb
findingprotoc
executable to better support cross compilation.Describe the pull request
What does your PR fix?
N/A
Which triplets are supported/not supported? Have you updated the CI baseline?
Yes
Does your PR follow the maintainer guide?
I believe so, yes
If you have added/updated a port: Have you run
./vcpkg x-add-version --all
and committed the result?Yes
If you are still working on the PR, open it as a Draft: https://github.blog/2019-02-14-introducing-draft-pull-requests/