protobuf: allow shared on Macos + add protobuf::protoc imported target #4776

SpaceIm · 2021-03-03T19:02:32Z

Specify library name and version: lib/1.0

I've read the guidelines for contributing.
I've followed the PEP8 style guides for Python code in the recipes.
I've used the latest Conan client version.
I've tried at least one configuration locally with the
conan-center hook activated.

Modifications in protobuf-config.cmake.in were too fragile. They break if with_zlib=True due to expansion of CMake variable with space separators.
This PR also provides a real protobuf::protoc imported target.

previous injection of lib directories was too fragile. It breaks as soon as there is a dependency

ghost · 2021-03-03T19:13:40Z

I detected other pull requests that are modifying protobuf/all recipe:

[protobuf] Enable Android/iOS cross building #4556

This message is automatically generated by https://github.com/ericLemanissier/conan-center-conflicting-prs so don't hesitate to report issues/improvements there.

conan-center-bot · 2021-03-03T19:53:58Z

Some configurations of 'protobuf/3.9.1' failed in build 1 (5df0f0892f1bd2787517708b876ec51a621b0c8a):

Macos x86_64, Release, apple-clang 9.1, libc++ . Options: protobuf:shared-True
Access to all the logs
Macos x86_64, Release, apple-clang 10.0, libc++ . Options: protobuf:shared-True
Access to all the logs

Croydon · 2021-03-03T20:04:31Z

recipes/protobuf/all/conanfile.py

@@ -48,20 +42,23 @@ def configure(self):
        if self.options.shared:
            del self.options.fPIC

-            if self.settings.os == "Windows" and self.settings.compiler in ["Visual Studio", "clang"] and "MT" in self.settings.compiler.runtime:
+            if self.settings.compiler.get_safe("runtime") in ["MT", "MTd"]:


Not sure if we already care about this, but this seems not to be future proof with the settings for the new MSVC compiler

https://github.com/conan-io/conan/blob/03b3d71890c537df1154b3d04cc35ed2654a5894/conans/client/conf/__init__.py#L87

Not a regression in any case, as the old code did not support the MSVC compiler settings either

I can add "static". A new method in tools would be welcome to avoid fragile verifications (something like tools.is_static_runtime(self.settings.compiler). There are many recipes like that.

conan-center-bot · 2021-03-03T21:32:14Z

Some configurations of 'protobuf/3.9.1' failed in build 2 (0913f5fab9641476a393685ec3cd5e6050d0fdb7):

Macos x86_64, Release, apple-clang 9.1, libc++ . Options: protobuf:shared-True
Access to all the logs
Macos x86_64, Release, apple-clang 10.0, libc++ . Options: protobuf:shared-True
Access to all the logs

conan-center-bot · 2021-03-03T23:04:35Z

Some configurations of 'protobuf/3.9.1' failed in build 3 (725d9d4b09104cc61bbceccf371c01d357c31c21):

Macos x86_64, Release, apple-clang 9.1, libc++ . Options: protobuf:shared-True
Access to all the logs

SpaceIm · 2021-03-03T23:23:18Z

dyld: Library not loaded: libprotoc.3.9.1.0.dylib

Why? I can't reproduce this error on my Mac.

conan-center-bot · 2021-03-04T00:17:02Z

Some configurations of 'protobuf/3.9.1' failed in build 4 (f8fb68fe5e4024dcaa0efec28553ca8a9905cbfd):

Macos x86_64, Release, apple-clang 9.1, libc++ . Options: protobuf:shared-True
Access to all the logs
Macos x86_64, Release, apple-clang 10.0, libc++ . Options: protobuf:shared-True
Access to all the logs

conan-center-bot · 2021-03-04T10:26:01Z

Some configurations of 'protobuf/3.9.1' failed in build 5 (94b882011ed3fc0ce75a2d18de8f4a7b68fec42c):

Macos x86_64, Release, apple-clang 9.1, libc++ . Options: protobuf:shared-True
Access to all the logs
Macos x86_64, Release, apple-clang 10.0, libc++ . Options: protobuf:shared-True
Access to all the logs

SpaceIm · 2021-03-04T12:38:11Z

@avantgardnerio does it break the way you use protoc (#4012)? I guess it was a workaround for cross compilation? If it breaks, I can add find_program again and set IMPORTED_LOCATION property of protobuf::protoc imported target with this value.

conan-center-bot · 2021-03-19T15:45:26Z

All green in build 24 (a46fe2d549b60a83fc28026a93051cc0c783f60d):

protobuf/3.9.1@:
All packages built successfully! (All logs)
protobuf/3.11.4@:
All packages built successfully! (All logs)
protobuf/3.12.4@:
All packages built successfully! (All logs)
protobuf/3.15.5@:
All packages built successfully! (All logs)
protobuf/3.13.0@:
All packages built successfully! (All logs)

anton-danielsson · 2021-03-19T17:08:25Z

recipes/protobuf/all/conanfile.py

-# Find the protobuf compiler within the paths added by Conan, for use below.
-find_program(PROTOC_PROGRAM protoc PATHS ENV PATH NO_DEFAULT_PATH)
-if(NOT PROTOC_PROGRAM)
-    set(PROTOC_PROGRAM "protoc")


It's not clear to me how this is not conflicting with:
369121c#diff-5dfb647bf8fb82769ddb94143d6b47475c5d4d2db2346115ee6b1991a6db6445R112
What will the merge look like?

It's removed. This PR adds protobuf::protoc imported target and Protobuf_PROTOC_EXECUTABLE CACHE variable, but it's not very clear for me if it works with two profiles. I would like some feedbacks.
I should turn this PR to draft for the moment, since it could break cross compilation.

I have pasted some commands below with one way you can test it.
The dual profile build doesn't seem to work.
CMake will find the wrong protoc (the one from host context instead of the one from build context).

Command:

conan create . 3.15.5@foo/testing \ --profile:host android-ndk-r22-api-21-arm64-v8a-clang-c++_static.txt \ --profile:build default \ --build=protobuf

android-ndk-r22-api-21-arm64-v8a-clang-c++_static.txt:

[settings] arch=armv8 build_type=Release compiler=clang compiler.libcxx=c++_static compiler.version=11 os=Android os.api_level=21 [build_requires] android-ndk/r22 [options] [env]

Result:

protobuf/3.15.5@foo/testing (test package): Calling build() ... [1/4] Running cpp protocol buffer compiler on addressbook.proto FAILED: addressbook.pb.h addressbook.pb.cc ... /bin/sh: 1: x/.conan/data/protobuf/3.15.5/dirac/testing/package/475278d37419b9e365c649e594b7c9a419a343bb/bin/protoc: Exec format error

Ok so maybe we could keep this find_program, and use this path for protobuf::protoc properties.

Yeah, could work. I'm happy to try if you push something later :)

Could you test with a0f4c76 please?

Seems to work just fine. I'm happy with it :)

conan-center-bot · 2021-03-21T14:12:28Z

Failure in build 25 (a0f4c761d7f44167961a272a407d215a439e086c):

protobuf/3.9.1@:
An unexpected error happened and has been reported
protobuf/3.12.4@:
All packages built successfully! (All logs)
protobuf/3.13.0@:
All packages built successfully! (All logs)
protobuf/3.11.4@:
All packages built successfully! (All logs)
protobuf/3.15.5@:
Didn't run or was cancelled before finishing

^{Note: To save resources, CI tries to finish as soon as an error is found. For this reason you might find that not all the references have been launched or not all the configurations for a given reference. Also, take into account that we cannot guarantee the order of execution as it depends on CI workload and workers availability.}

conan-center-bot · 2021-03-21T15:15:56Z

All green in build 26 (a0f4c761d7f44167961a272a407d215a439e086c):

protobuf/3.9.1@:
All packages built successfully! (All logs)
protobuf/3.11.4@:
All packages built successfully! (All logs)
protobuf/3.12.4@:
All packages built successfully! (All logs)
protobuf/3.13.0@:
All packages built successfully! (All logs)
protobuf/3.15.5@:
All packages built successfully! (All logs)

conan-center-bot · 2021-03-22T13:04:42Z

All green in build 27 (a0f4c761d7f44167961a272a407d215a439e086c):

protobuf/3.9.1@:
All packages built successfully! (All logs)
protobuf/3.11.4@:
All packages built successfully! (All logs)
protobuf/3.12.4@:
All packages built successfully! (All logs)
protobuf/3.13.0@:
All packages built successfully! (All logs)
protobuf/3.15.5@:
All packages built successfully! (All logs)

zuowanbushiwo · 2021-08-29T03:04:39Z

I have some troubles when cross-compiling

I follow conan doc Linux/Windows/macOS to Android , create a android profile host， as follow：


include(default)

target_host=aarch64-linux-android
api_level=24

[settings]
arch=armv8
build_type=Release
compiler=clang
compiler.libcxx=libc++
compiler.version=11
os=Android
os.api_level=$api_level

[build_requires]

[options]

[conf]
tools.android:ndk_path=/opt/android-sdk/ndk-bundle

[env]
CONAN_CMAKE_TOOLCHAIN_FILE=/opt/android-sdk/ndk-bundle/build/cmake/android.toolchain.cmake
PATH=[/opt/android-sdk/ndk-bundle/toolchains/llvm/prebuilt/linux-x86_64/bin]
CHOST=$target_host
AR=$target_host-ar
AS=$target_host-as
RANLIB=$target_host-ranlib
CC=$target_host$api_level-clang
CXX=$target_host$api_level-clang++
LD=$target_host-ld
STRIP=$target_host-strip

I use the following command follow document guide:
conan-center-index/recipes/protobuf/all$ conan create . 3.15.5@ -pr:h ~/conan_exsample/conan-center-index/recipes/android -pr:b default --build

build depends protobuf and zib success， build test package， A major error occurred：


CMake Error at CMakeLists.txt:7 (find_package):
  Could not find a package configuration file provided by "protobuf" with any
  of the following names:

    protobufConfig.cmake
    protobuf-config.cmake

  Add the installation prefix of "protobuf" to CMAKE_PREFIX_PATH or set
  "protobuf_DIR" to a directory containing one of the above files.  If
  "protobuf" provides a separate development package or SDK, be sure it has
  been installed.


-- Configuring incomplete, errors occurred!
See also "/home/packager/conan_exsample/conan-center-index/recipes/protobuf/all/test_package/build/231d0c73cb5a9d0b3a61fdaeff116c5c0990ae0e/CMakeFiles/CMakeOutput.log".
ERROR: protobuf/3.15.5 (test package): Error in build() method, line 16
	cmake.configure()
	ConanException: Error 1 while executing cd '/home/packager/conan_exsample/conan-center-index/recipes/protobuf/all/test_package/build/231d0c73cb5a9d0b3a61fdaeff116c5c0990ae0e' && cmake -G "Unix Makefiles" -DCMAKE_BUILD_TYPE="Release" -DCMAKE_TOOLCHAIN_FILE="/opt/android-sdk/ndk-bundle/build/cmake/android.toolchain.cmake" -DCONAN_IN_LOCAL_CACHE="OFF" -DCONAN_COMPILER="clang" -DCONAN_COMPILER_VERSION="11" -DCONAN_LIBCXX="libc++" -DCMAKE_INSTALL_PREFIX="/home/packager/conan_exsample/conan-center-index/recipes/protobuf/all/test_package/build/231d0c73cb5a9d0b3a61fdaeff116c5c0990ae0e/package" -DCMAKE_INSTALL_BINDIR="bin" -DCMAKE_INSTALL_SBINDIR="bin" -DCMAKE_INSTALL_LIBEXECDIR="bin" -DCMAKE_INSTALL_LIBDIR="lib" -DCMAKE_INSTALL_INCLUDEDIR="include" -DCMAKE_INSTALL_OLDINCLUDEDIR="include" -DCMAKE_INSTALL_DATAROOTDIR="share" -DCMAKE_PREFIX_PATH="/home/packager/conan_exsample/conan-center-index/recipes/protobuf/all/test_package/build/231d0c73cb5a9d0b3a61fdaeff116c5c0990ae0e" -DCMAKE_MODULE_PATH="/home/packager/conan_exsample/conan-center-index/recipes/protobuf/all/test_package/build/231d0c73cb5a9d0b3a61fdaeff116c5c0990ae0e" -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY="ON" -DCONAN_EXPORTED="1" -Dprotobuf_LITE="False" -Wno-dev '/home/packager/conan_exsample/conan-center-index/recipes/protobuf/all/test_package

The contents of the build directory are as follows:

prince-chrismc · 2021-08-29T14:58:55Z

I am no expert but, I've seen a different profile being used... #4220 has a ton of information you might be intereseted in... also #4224 points to an older example... also #4556 added support and OP left a lot of notes

I'd suggestion opening an issue if you need more help 👍

zuowanbushiwo · 2021-08-31T04:52:45Z

Thanks for that interesting links ! That was all new to me. I will read and understand it carefully.

SpaceIm added 5 commits March 3, 2021 18:53

really provide protobuf::protoc target

f320810

cosmetic change

f13d5b7

simplify check of runtime

d636224

properly handle protobuf with zlib

c4037a2

previous injection of lib directories was too fragile. It breaks as soon as there is a dependency

append to builddirs

5df0f08

ghost mentioned this pull request Mar 3, 2021

[service] Potentially conflicting recipes #3571

Closed

This was referenced Mar 3, 2021

feat(protobuf): Add an option to remove RTTI from binaries #4682

Merged

[protobuf] Enable Android/iOS cross building #4556

Merged

github-actions bot mentioned this pull request Mar 3, 2021

Pull Requests Ready to be Reviewed prince-chrismc/conan-center-index-pending-review#1

Open

cleanup

0913f5f

conan-center-bot added the Failed label Mar 3, 2021

Croydon reviewed Mar 3, 2021

View reviewed changes

fix semicolon replacement

725d9d4

more robust DYLD_LIBRARY_PATH injection

d879ac2

SpaceIm force-pushed the fix/protobu-protoc-target branch from 94b8820 to d879ac2 Compare March 4, 2021 11:43

SpaceIm added 5 commits March 4, 2021 14:28

cosmetic change

c2c6033

improve runtime handling

ab34548

no protobuf_BUILD_LIBPROTOC option before 3.14.0

0408def

honor upstream default value for with_zlib

f5a965d

add protobuf/3.15.4

a7d6760

SpaceIm changed the title ~~protobuf: allow shared on Macos and improve robustness~~ protobuf: add 3.15.4 + allow shared on Macos + improve protoc call robustness Mar 4, 2021

provide Protobuf_PROTOC_EXECUTABLE CACHE variable

10ac43a

empty commit

a46fe2d

conan-center-bot removed Unexpected Error Failed labels Mar 19, 2021

anton-danielsson reviewed Mar 19, 2021

View reviewed changes

SpaceIm marked this pull request as draft March 19, 2021 18:03

keep find_program protoc logic for cross compilation

a0f4c76

SpaceIm marked this pull request as ready for review March 21, 2021 13:24

conan-center-bot added Failed Unexpected Error labels Mar 21, 2021

SpaceIm closed this Mar 21, 2021

SpaceIm reopened this Mar 21, 2021

conan-center-bot removed Unexpected Error Failed labels Mar 21, 2021

SpaceIm requested a review from uilianries March 21, 2021 16:49

prince-chrismc approved these changes Mar 21, 2021

View reviewed changes

SpaceIm changed the title ~~protobuf: add 3.15.5 + allow shared on Macos + improve protoc call robustness~~ protobuf: allow shared on Macos + add protobuf::protoc imported target Mar 22, 2021

uilianries approved these changes Mar 22, 2021

View reviewed changes

SSE4 approved these changes Mar 23, 2021

View reviewed changes

conan-center-bot merged commit 7cf61ab into conan-io:master Mar 23, 2021

SpaceIm deleted the fix/protobu-protoc-target branch March 23, 2021 07:46

This was referenced Mar 25, 2021

[package] protobuf: can't mimic correct CMake variables from upstream #3492

Closed

Enable cross building for Android and iOS inexorgame-obsolete/conan-grpc#51

Closed

SpaceIm mentioned this pull request Mar 28, 2021

onnx: bump protobuf #5082

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

protobuf: allow shared on Macos + add protobuf::protoc imported target #4776

protobuf: allow shared on Macos + add protobuf::protoc imported target #4776

SpaceIm commented Mar 3, 2021

ghost commented Mar 3, 2021 •

edited by ghost

Loading

conan-center-bot commented Mar 3, 2021

Croydon Mar 3, 2021

Croydon Mar 3, 2021

SpaceIm Mar 3, 2021 •

edited

Loading

conan-center-bot commented Mar 3, 2021

conan-center-bot commented Mar 3, 2021

SpaceIm commented Mar 3, 2021 •

edited

Loading

conan-center-bot commented Mar 4, 2021

conan-center-bot commented Mar 4, 2021

SpaceIm commented Mar 4, 2021 •

edited

Loading

conan-center-bot commented Mar 19, 2021

anton-danielsson Mar 19, 2021

SpaceIm Mar 19, 2021 •

edited

Loading

anton-danielsson Mar 19, 2021

SpaceIm Mar 19, 2021

anton-danielsson Mar 19, 2021

SpaceIm Mar 21, 2021

anton-danielsson Mar 21, 2021

conan-center-bot commented Mar 21, 2021

conan-center-bot commented Mar 21, 2021

conan-center-bot commented Mar 22, 2021

zuowanbushiwo commented Aug 29, 2021

prince-chrismc commented Aug 29, 2021

zuowanbushiwo commented Aug 31, 2021

protobuf: allow shared on Macos + add protobuf::protoc imported target #4776

protobuf: allow shared on Macos + add protobuf::protoc imported target #4776

Conversation

SpaceIm commented Mar 3, 2021

ghost commented Mar 3, 2021 • edited by ghost Loading

conan-center-bot commented Mar 3, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

SpaceIm Mar 3, 2021 • edited Loading

Choose a reason for hiding this comment

conan-center-bot commented Mar 3, 2021

conan-center-bot commented Mar 3, 2021

SpaceIm commented Mar 3, 2021 • edited Loading

conan-center-bot commented Mar 4, 2021

conan-center-bot commented Mar 4, 2021

SpaceIm commented Mar 4, 2021 • edited Loading

conan-center-bot commented Mar 19, 2021

Choose a reason for hiding this comment

SpaceIm Mar 19, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

conan-center-bot commented Mar 21, 2021

conan-center-bot commented Mar 21, 2021

conan-center-bot commented Mar 22, 2021

zuowanbushiwo commented Aug 29, 2021

prince-chrismc commented Aug 29, 2021

zuowanbushiwo commented Aug 31, 2021

ghost commented Mar 3, 2021 •

edited by ghost

Loading

SpaceIm Mar 3, 2021 •

edited

Loading

SpaceIm commented Mar 3, 2021 •

edited

Loading

SpaceIm commented Mar 4, 2021 •

edited

Loading

SpaceIm Mar 19, 2021 •

edited

Loading