ARROW-11204: [C++] Fix build failures with bundled gRPC and Protobuf #9157

kou · 2021-01-10T22:17:00Z

This is caused by ARROW-9400, 0ed34c8 .

There are some problems:

"-Dprotobuf_MSVC_STATIC_RUNTIME=" (empty value) is a problem with non MSVC
Can't find RE2 with CMake 3.11 or older

A failure log (no useful information):

https://github.com/ursa-labs/crossbow/runs/1675885130#step:5:877

FAILED: grpc_ep-prefix/src/grpc_ep-stamp/grpc_ep-configure
cd /.../grpc_ep-prefix/src/grpc_ep-build &&
cmake -P /.../grpc_ep-prefix/src/grpc_ep-stamp/grpc_ep-configure-RELEASE.cmake &&
cmake -E touch /.../grpc_ep-prefix/src/grpc_ep-stamp/grpc_ep-configure
CMake Error at /.../grpc_ep-prefix/src/grpc_ep-stamp/grpc_ep-configure-RELEASE.cmake:16 (message):
  Command failed: 1

   '/usr/bin/cmake'
   '-DCMAKE_C_COMPILER=/usr/bin/cc'
   '-DCMAKE_CXX_COMPILER=/usr/bin/c++'
   '-DCMAKE_AR=/usr/bin/ar'
   '-DCMAKE_RANLIB=/usr/bin/ranlib'
   '-DCMAKE_BUILD_TYPE=RELEASE'
   '-DCMAKE_C_FLAGS=...'
   '-DCMAKE_C_FLAGS_RELEASE=...'
   '-DCMAKE_CXX_FLAGS=...'
   '-DCMAKE_CXX_FLAGS_RELEASE=...'
   '-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY='
   '-DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY='
   '-DCMAKE_PREFIX_PATH=';/.../protobuf_ep-install;;/.../cares_ep-install;;/.../absl_ep-install''
   '-DgRPC_ABSL_PROVIDER=package'
   '-DgRPC_BUILD_CSHARP_EXT=OFF'
   '-DgRPC_BUILD_GRPC_CSHARP_PLUGIN=OFF'
   '-DgRPC_BUILD_GRPC_NODE_PLUGIN=OFF'
   '-DgRPC_BUILD_GRPC_OBJECTIVE_C_PLUGIN=OFF'
   '-DgRPC_BUILD_GRPC_PHP_PLUGIN=OFF'
   '-DgRPC_BUILD_GRPC_PYTHON_PLUGIN=OFF'
   '-DgRPC_BUILD_GRPC_RUBY_PLUGIN=OFF'
   '-DgRPC_BUILD_TESTS=OFF'
   '-DgRPC_CARES_PROVIDER=package'
   '-DgRPC_GFLAGS_PROVIDER=package'
   '-DgRPC_MSVC_STATIC_RUNTIME='
   '-DgRPC_PROTOBUF_PROVIDER=package'
   '-DgRPC_PROTOBUF_PACKAGE_TYPE=CONFIG'
   '-DgRPC_RE2_PROVIDER=package'
   '-DgRPC_SSL_PROVIDER=package'
   '-DgRPC_ZLIB_PROVIDER=package'
   '-DgRPC_MSVC_STATIC_RUNTIME='
   '-DCMAKE_INSTALL_PREFIX=/.../grpc_ep-install'
   '-DCMAKE_INSTALL_LIBDIR=lib'
   '-DBUILD_SHARED_LIBS=OFF'
   '-GNinja'
   '/.../grpc_ep-prefix/src/grpc_ep'

  See also

    /.../grpc_ep-prefix/src/grpc_ep-stamp/grpc_ep-configure-*.log

"-Dprotobuf_MSVC_STATIC_RUNTIME=" (empty value) is a problem on this case. This is caused by ARROW-9400, 0ed34c8 .

github-actions · 2021-01-10T22:17:21Z

https://issues.apache.org/jira/browse/ARROW-11204

kou · 2021-01-10T22:18:42Z

The changes in dev/tasks/linux-packages/package-task.rb aren't related to this fix.
The changes are convenient to debug this problem in reproduced environment (RPM build environment in this case).

kou · 2021-01-10T22:19:47Z

@github-actions crossbow submit -g linux

github-actions · 2021-01-10T22:21:00Z

Revision: b3d1418

Submitted crossbow builds: ursa-labs/crossbow @ actions-866

Task	Status
centos-7-aarch64
centos-7-amd64
centos-8-aarch64
centos-8-amd64
debian-buster-amd64
debian-buster-arm64
ubuntu-bionic-amd64
ubuntu-bionic-arm64
ubuntu-focal-amd64
ubuntu-focal-arm64
ubuntu-groovy-amd64
ubuntu-groovy-arm64
ubuntu-xenial-amd64
ubuntu-xenial-arm64

kou · 2021-01-11T00:38:52Z

cpp/cmake_modules/ThirdpartyToolchain.cmake

      -DCMAKE_INSTALL_PREFIX=${GRPC_PREFIX}
      -DCMAKE_INSTALL_LIBDIR=lib
      -DBUILD_SHARED_LIBS=OFF)
-  if(RE2_VENDORED)
-    list(APPEND GRPC_CMAKE_ARGS -Dre2_ROOT=${RE2_PREFIX}


We can't use <PackageName>_ROOT with CMake 3.11 or older.
(This is not related to MSVC.)

kou · 2021-01-11T00:39:05Z

@github-actions crossbow submit -g linux

github-actions · 2021-01-11T00:40:00Z

Revision: 218829a

Submitted crossbow builds: ursa-labs/crossbow @ actions-867

Task	Status
centos-7-aarch64
centos-7-amd64
centos-8-aarch64
centos-8-amd64
debian-buster-amd64
debian-buster-arm64
ubuntu-bionic-amd64
ubuntu-bionic-arm64
ubuntu-focal-amd64
ubuntu-focal-arm64
ubuntu-groovy-amd64
ubuntu-groovy-arm64
ubuntu-xenial-amd64
ubuntu-xenial-arm64

kou · 2021-01-11T19:47:25Z

@github-actions crossbow submit centos-*-amd64

github-actions · 2021-01-11T22:25:00Z

Revision: 218829a

Submitted crossbow builds: ursa-labs/crossbow @ actions-872

Task	Status
centos-7-amd64
centos-8-amd64

kou · 2021-01-12T00:45:18Z

@github-actions crossbow submit centos-*-amd64

github-actions · 2021-01-12T01:10:08Z

Revision: 1806d48

Submitted crossbow builds: ursa-labs/crossbow @ actions-874

Task	Status
centos-7-amd64
centos-8-amd64

kou · 2021-01-12T06:43:44Z

+1
The Java JNI failure is unrelated: https://github.com/apache/arrow/pull/9157/checks?check_run_id=1685053738#step:17:1

Post job cleanup.
/bin/tar -cz -f /home/runner/work/_temp/2ff4a229-7a60-4949-af10-5e26fe774e4c/cache.tgz -C /home/runner/work/arrow/arrow/.docker .
Warning: read ECONNRESET
events.js:187
      throw er; // Unhandled 'error' event
      ^

Error: EBADF: bad file descriptor, read
Emitted 'error' event on ReadStream instance at:
    at internal/fs/streams.js:167:12
    at FSReqCallback.wrapper [as oncomplete] (fs.js:470:5) {
  errno: -9,
  code: 'EBADF',
  syscall: 'read'
}

This is caused by ARROW-9400, 0ed34c8 . There are some problems: * "-Dprotobuf_MSVC_STATIC_RUNTIME=" (empty value) is a problem with non MSVC * Can't find RE2 with CMake 3.11 or older A failure log (no useful information): https://github.com/ursa-labs/crossbow/runs/1675885130#step:5:877 FAILED: grpc_ep-prefix/src/grpc_ep-stamp/grpc_ep-configure cd /.../grpc_ep-prefix/src/grpc_ep-build && cmake -P /.../grpc_ep-prefix/src/grpc_ep-stamp/grpc_ep-configure-RELEASE.cmake && cmake -E touch /.../grpc_ep-prefix/src/grpc_ep-stamp/grpc_ep-configure CMake Error at /.../grpc_ep-prefix/src/grpc_ep-stamp/grpc_ep-configure-RELEASE.cmake:16 (message): Command failed: 1 '/usr/bin/cmake' '-DCMAKE_C_COMPILER=/usr/bin/cc' '-DCMAKE_CXX_COMPILER=/usr/bin/c++' '-DCMAKE_AR=/usr/bin/ar' '-DCMAKE_RANLIB=/usr/bin/ranlib' '-DCMAKE_BUILD_TYPE=RELEASE' '-DCMAKE_C_FLAGS=...' '-DCMAKE_C_FLAGS_RELEASE=...' '-DCMAKE_CXX_FLAGS=...' '-DCMAKE_CXX_FLAGS_RELEASE=...' '-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=' '-DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=' '-DCMAKE_PREFIX_PATH=';/.../protobuf_ep-install;;/.../cares_ep-install;;/.../absl_ep-install'' '-DgRPC_ABSL_PROVIDER=package' '-DgRPC_BUILD_CSHARP_EXT=OFF' '-DgRPC_BUILD_GRPC_CSHARP_PLUGIN=OFF' '-DgRPC_BUILD_GRPC_NODE_PLUGIN=OFF' '-DgRPC_BUILD_GRPC_OBJECTIVE_C_PLUGIN=OFF' '-DgRPC_BUILD_GRPC_PHP_PLUGIN=OFF' '-DgRPC_BUILD_GRPC_PYTHON_PLUGIN=OFF' '-DgRPC_BUILD_GRPC_RUBY_PLUGIN=OFF' '-DgRPC_BUILD_TESTS=OFF' '-DgRPC_CARES_PROVIDER=package' '-DgRPC_GFLAGS_PROVIDER=package' '-DgRPC_MSVC_STATIC_RUNTIME=' '-DgRPC_PROTOBUF_PROVIDER=package' '-DgRPC_PROTOBUF_PACKAGE_TYPE=CONFIG' '-DgRPC_RE2_PROVIDER=package' '-DgRPC_SSL_PROVIDER=package' '-DgRPC_ZLIB_PROVIDER=package' '-DgRPC_MSVC_STATIC_RUNTIME=' '-DCMAKE_INSTALL_PREFIX=/.../grpc_ep-install' '-DCMAKE_INSTALL_LIBDIR=lib' '-DBUILD_SHARED_LIBS=OFF' '-GNinja' '/.../grpc_ep-prefix/src/grpc_ep' See also /.../grpc_ep-prefix/src/grpc_ep-stamp/grpc_ep-configure-*.log Closes apache#9157 from kou/cpp-grpc-bundled Authored-by: Sutou Kouhei <kou@clear-code.com> Signed-off-by: Sutou Kouhei <kou@clear-code.com>

[C++] Fix build failure with bundled gRPC and Protobuf on non MSVC

b3d1418

"-Dprotobuf_MSVC_STATIC_RUNTIME=" (empty value) is a problem on this case. This is caused by ARROW-9400, 0ed34c8 .

github-actions bot added the Component: C++ label Jan 10, 2021

<PackageName>_ROOT isn't available CMake 3.11 or older

218829a

kou commented Jan 11, 2021

View reviewed changes

kou changed the title ~~ARROW-11204: [C++] Fix build failure with bundled gRPC and Protobuf on non MSVC~~ ARROW-11204: [C++] Fix build failures with bundled gRPC and Protobuf Jan 11, 2021

Fix a typo

1806d48

kou closed this in abdeb40 Jan 12, 2021

kou deleted the cpp-grpc-bundled branch January 12, 2021 06:45

asfimport mentioned this pull request Jan 12, 2021

[C++] Fix build failure with bundled gRPC and Protobuf #27109

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ARROW-11204: [C++] Fix build failures with bundled gRPC and Protobuf #9157

ARROW-11204: [C++] Fix build failures with bundled gRPC and Protobuf #9157

kou commented Jan 10, 2021 •

edited

github-actions bot commented Jan 10, 2021

kou commented Jan 10, 2021

kou commented Jan 10, 2021

github-actions bot commented Jan 10, 2021

kou Jan 11, 2021

kou commented Jan 11, 2021

github-actions bot commented Jan 11, 2021

kou commented Jan 11, 2021

github-actions bot commented Jan 11, 2021

kou commented Jan 12, 2021

github-actions bot commented Jan 12, 2021

kou commented Jan 12, 2021

ARROW-11204: [C++] Fix build failures with bundled gRPC and Protobuf #9157

ARROW-11204: [C++] Fix build failures with bundled gRPC and Protobuf #9157

Conversation

kou commented Jan 10, 2021 • edited

github-actions bot commented Jan 10, 2021

kou commented Jan 10, 2021

kou commented Jan 10, 2021

github-actions bot commented Jan 10, 2021

kou Jan 11, 2021

Choose a reason for hiding this comment

kou commented Jan 11, 2021

github-actions bot commented Jan 11, 2021

kou commented Jan 11, 2021

github-actions bot commented Jan 11, 2021

kou commented Jan 12, 2021

github-actions bot commented Jan 12, 2021

kou commented Jan 12, 2021

kou commented Jan 10, 2021 •

edited