CMake: add separate export for plugin targets

[herbrechtsmeier]

Add a separate export for the plugin targets to separate binaries and libraries into their own target export files. Replace the cross binary plugin targets with native binary targets during cross
compile to allow the usage of the plugin targets for native and cross compile builds.

@markdroth

[linux-foundation-easycla]

The committers are authorized under a signed CLA.

• ✅ herbrechtsmeier (9ecb8a5, d140c34)

[herbrechtsmeier]

I signed it

[markdroth]

Looks like the CLA bot is still failing. If you're having problems getting your signature to register, please file a support ticket using the link shown above. Thanks!

[jtattermusch]

I'm not sure I understand how this is useful and what's the use case. Can you give an example?

[jtattermusch]

Why do we need this change?

[herbrechtsmeier]

Names of variables used internally should start with an underscore. Because this variable is used inside the gRPCConfig.cmake.in and the name use the project namespace I have remove it.

[herbrechtsmeier]

During cross compile you have to combine the cross compiled libraries with the native binaries. At the moment every downstream project have to find the program by itself:

if(CMAKE_CROSSCOMPILING)
    find_program(_GRPC_CPP_PLUGIN_EXECUTABLE grpc_cpp_plugin)
  else()
    set(_GRPC_CPP_PLUGIN_EXECUTABLE $<TARGET_FILE:grpc_cpp_plugin>)
  endif()

After this patch you could always use $<TARGET_FILE:grpc_cpp_plugin>.

[herbrechtsmeier]

@markdroth According to the contributing guidelines we have sign the Contributor License Agreement but this isn't respected by the EasyCLA.

[herbrechtsmeier]

@jtattermusch At the moment the gRPC CMake support doesn't work in Yocto because Yocto doesn't install the target binaries into the target sysroot and this is required by the gRPCConfig.cmake. The current solution is to remove this file from the target sysroot but this breaks other projects which use the library targets because CMake finds the gRPCConfig.cmake in the native sysroot and thereby links against the native libraries instead of the target libraries.

[herbrechtsmeier]

We signed the EasyCLA in addition to the CLA from the CONTRIBUTING.md.

[stale]

This issue/PR has been automatically marked as stale because it has not had any update (including commits, comments, labels, milestones, etc) for 30 days. It will be closed automatically if no further update occurs in 7 day. Thank you for your contributions!

[herbrechtsmeier]

What can I do to get this reviewed?

[herbrechtsmeier]

@jtattermusch Yes. You have to apply this changes and remove the SYSROOT_DIRS_BLACKLIST_append_class-target hack.

[mmurrian]

@jtattermusch Yes. You have to apply this changes and remove the SYSROOT_DIRS_BLACKLIST_append_class-target hack.

Thanks for the quick response. I deleted my post because I realized I still had the hack you mentioned in place.

This has been on my back-burner for a while. I circled back around and you already fixed it for me!

[herbrechtsmeier]

@jtattermusch Yes. You have to apply this changes and remove the SYSROOT_DIRS_BLACKLIST_append_class-target hack.

Thanks for the quick response. I deleted my post because I realized I still had the hack you mentioned in place.

This has been on my back-burner for a while. I circled back around and you already fixed it for me!

The hack isn't from me and it doesn't work correct. You couldn't use the grpc:: targets for linking because they point to the native libraries.

[jtattermusch]

Based on my review, it seems that this PR doesn't actually fix anything, it just adds some syntactic sugar.
I have some doubts whether what the PR tries to do is correct and because it's not a bugfix, i'll close.
Sorry.

[jtattermusch]

this really seems just as a convenience, I don't think it enables anything that wasn't possible before.
As such, I don't think we want this improvement due to its complexity.

It looks like all the other changes have been added just to be able to add this snippet, so IMHO also the other changes from this PR changes aren't really needed.

[herbrechtsmeier]

this really seems just as a convenience, I don't think it enables anything that wasn't possible before.

At the moment every project inclusive the grpc examples have to check the variable CMAKE_CROSSCOMPILING to find the plugin grpc_cpp_plugin or to use the target grpc::grpc_cpp_plugin:

if(CMAKE_CROSSCOMPILING)
  find_program(_GRPC_CPP_PLUGIN_EXECUTABLE grpc_cpp_plugin)
else()
  set(_GRPC_CPP_PLUGIN_EXECUTABLE $<TARGET_FILE:grpc_cpp_plugin>)
endif()

After the patch every project could simple use the target.

This

As such, I don't think we want this improvement due to its complexity.

This pull request moves the complexity from every project into the source project and thereby remove code duplication.

It looks like all the other changes have been added just to be able to add this snippet, so IMHO also the other changes from this PR changes aren't really needed.

Which changes do you mean?

The PR segregate the host tools from the libraries and search for the host tools during a cross compile.

Please check your examples. At the moment you force everybody to copy code into its CMakeLists.txt.

[mmurrian]

One such "convenience" is the ability to compile and use gRPC in Yocto.

Cross-compiling with gRPC is already unique given the need for both "native" and "target" phases of compilation.

Other libraries (say, Qt) go so far as to have special functionality (AUTOMOC) added to cmake in support of code generation.

gRPC can't even accommodate one additional build target?

[jtattermusch]

One such "convenience" is the ability to compile and use gRPC in Yocto.

Cross-compiling with gRPC is already unique given the need for both "native" and "target" phases of compilation.

Other libraries (say, Qt) go so far as to have special functionality (AUTOMOC) added to cmake in support of code generation.

gRPC can't even accommodate one additional build target?

As said before, Yocto is not a platform we officially support. That said, given the nature of the proposed change I think there should still be a way to build with gRPC under yocto (maybe not in the most straighforward way, but none of the changes you're proposing seem to be strictly necessary).

gRPC can accomodate one additional build target, but the need for such target needs to be clearly established and there needs to be a good way of testing this - otherwise increasing the complexity of the build system (which is already pretty complex) is harmful.

[herbrechtsmeier]

One such "convenience" is the ability to compile and use gRPC in Yocto.
Cross-compiling with gRPC is already unique given the need for both "native" and "target" phases of compilation.
Other libraries (say, Qt) go so far as to have special functionality (AUTOMOC) added to cmake in support of code generation.
gRPC can't even accommodate one additional build target?

As said before, Yocto is not a platform we officially support.

You don't support cross compiling. For example the run_distrib_test_raspberry_pi.sh only works because you install the native grpc tools into your root system.

That said, given the nature of the proposed change

What do you mean by this?

I think there should still be a way to build with gRPC under yocto (maybe not in the most straighforward way, but none of the changes you're proposing seem to be strictly necessary).

The problem is that every project need to find the plugin during cross compile. Instead of using the target grpc::grpc_cpp_plugin every project need the code from above.

gRPC can accomodate one additional build target, but the need for such target needs to be clearly established and there needs to be a good way of testing this

I could add a test if this is the problem. If we remove the above code from the example we could use them to test the changes.

• otherwise increasing the complexity of the build system (which is already pretty complex) is harmful.

It moves the complexity from the downstream projects to the root project.

[herbrechtsmeier]

The current solution with the find_program only supports cross compiling if the native program is installed inside the base system. It doesn't work if you have a sysroot for the native tools and cross libraries. CMake only support one list of roots (CMAKE_FIND_ROOT_PATH). A solution is to remove all binaries from the cross sysroot and list the native sysroot after the cross sysroot inside the variable CMAKE_FIND_ROOT_PATH. But this breaks the gRPCConfig.cmake because gRPC exports binaries and the gRPCTargets.cmake checks if all targets exist.

@jtattermusch What is the grpc solution for cross compiling with a native sysroot?

[clementperon]

@jtattermusch I get the same issue on my side.

My rootfs host is different from the rootfs target.

This make an error with the actual gRPCTargets.cmake because the cmake can't found the grpc_cpp_plugin in the "rootfs_target" because it's in the "rootfs_host".

IMO this is a proper solution to split the Tools/Plugins required for gRPC compilation vs the libs required for execution.

Regards,
Clement

[clementperon]

@veblush @gnossen @ashithasantosh,

At the moment the gRPC CMake doesn't support proper Cross compilation, what is done actually is to install both cross compiled libraries with the native binaries. Which is a big HACK! Merging different architecture in the same Rootfs doesn't work! If you want to have both host gRPC compiler and target gRPC compiler you can't!

I test this patch with host compilation (tools and bin are on x86_64), cross compilation (tools on x86_64 target on ARM64) and generating an SDK then cross-compiling gRPC files from host to a target (1.36.0).

All of them works properly, and splitting the gRPC targets from the gRPC lib/bin is the only proper way to make this cross compilation works properly.

Can this be review again, Please?

Thanks,
Clement