Cannot build google-cloud-cpp with bazel 0.29rc4

[johnedmonds]

ATTENTION! Please read and follow:

• if this is a question about how to build / test / query / deploy using Bazel, or a discussion starter, send it to bazel-discuss@googlegroups.com
• if this is a bug or feature request, fill the form below as best as you can.

Description of the problem / feature request:

Attempting to run ~/Downloads/bazel-0.29.0rc4-darwin-x86_64 build //google/cloud/bigtable:bigtable_client --incompatible_depset_is_not_iterable=false at https://github.com/googleapis/google-cloud-cpp/tree/5ad364581ed52ce457a2f79449964063e96242c6 results in

ERROR: /private/var/tmp/_bazel_jedmonds/22048c7097941053f1f6c951591631cf/external/com_google_googleapis/google/longrunning/BUILD.bazel:107:1: Action external/com_google_googleapis/google/longrunning/operations.grpc.pb.h failed (Exit 1) protoc failed: error executing command 
  (cd /private/var/tmp/_bazel_jedmonds/22048c7097941053f1f6c951591631cf/sandbox/darwin-sandbox/402/execroot/com_github_googleapis_google_cloud_cpp && \
  exec env - \
  bazel-out/host/bin/external/com_google_protobuf/protoc '--plugin=protoc-gen-PLUGIN=bazel-out/host/bin/external/com_github_grpc_grpc/grpc_cpp_plugin' '--PLUGIN_out=:bazel-out/darwin-fastbuild/bin/external/com_google_googleapis' '--proto_path=external/com_google_googleapis' '--proto_path=external/com_google_googleapis' '--proto_path=external/com_google_googleapis' '--proto_path=bazel-out/darwin-fastbuild/bin/external/com_google_protobuf' '--proto_path=external/com_google_googleapis' '--proto_path=bazel-out/darwin-fastbuild/bin/external/com_google_protobuf' '--proto_path=bazel-out/darwin-fastbuild/bin/external/com_google_protobuf' '--proto_path=bazel-out/darwin-fastbuild/bin/external/com_google_protobuf' '--proto_path=bazel-out/darwin-fastbuild/bin/external/com_google_googleapis/external/com_google_googleapis' external/com_google_googleapis/google/longrunning/operations.proto)
Execution platform: @bazel_tools//platforms:host_platform

Use --sandbox_debug to see verbose messages from the sandbox
bazel-out/darwin-fastbuild/bin/external/com_google_googleapis/external/com_google_googleapis: warning: directory does not exist.
google/protobuf/descriptor.proto: File not found.
google/api/annotations.proto: Import "google/protobuf/descriptor.proto" was not found or had errors.
google/api/annotations.proto:28:8: "google.protobuf.MethodOptions" is not defined.
google/protobuf/any.proto: File not found.
google/protobuf/duration.proto: File not found.
google/protobuf/empty.proto: File not found.
google/rpc/status.proto: Import "google/protobuf/any.proto" was not found or had errors.
google/rpc/status.proto:93:12: "google.protobuf.Any" is not defined.
google/longrunning/operations.proto: Import "google/api/annotations.proto" was not found or had errors.
google/longrunning/operations.proto: Import "google/protobuf/any.proto" was not found or had errors.
google/longrunning/operations.proto: Import "google/protobuf/duration.proto" was not found or had errors.
google/longrunning/operations.proto: Import "google/protobuf/empty.proto" was not found or had errors.
google/longrunning/operations.proto: Import "google/rpc/status.proto" was not found or had errors.
google/longrunning/operations.proto: Import "google/protobuf/descriptor.proto" was not found or had errors.
google/longrunning/operations.proto:132:3: "google.protobuf.Any" is not defined.
google/longrunning/operations.proto:144:5: "google.rpc.Status" is not defined.
google/longrunning/operations.proto:154:5: "google.protobuf.Any" is not defined.
google/longrunning/operations.proto:208:3: "google.protobuf.Duration" is not defined.
google/longrunning/operations.proto:35:8: "google.protobuf.MethodOptions" is not defined.
google/longrunning/operations.proto:84:56: "google.protobuf.Empty" is not defined.
google/longrunning/operations.proto:100:56: "google.protobuf.Empty" is not defined.
Target //google/cloud/bigtable:bigtable_client failed to build
INFO: Elapsed time: 0.547s, Critical Path: 0.10s
INFO: 2 processes: 2 darwin-sandbox.
FAILED: Build did NOT complete successfully

Bazel 0.28.0 succeeds

Feature requests: what underlying problem are you trying to solve with this feature?

Replace this line with your answer.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

Clone https://github.com/googleapis/google-cloud-cpp @ 5ad364581ed52ce457a2f79449964063e96242c6

Run ~/Downloads/bazel-0.29.0rc4-darwin-x86_64 build //google/cloud/bigtable:bigtable_client --incompatible_depset_is_not_iterable=false

What operating system are you running Bazel on?

OSX

What's the output of bazel info release?

release 0.29.0rc4

If bazel info release returns "development version" or "(@non-git)", tell us how you built Bazel.

Replace this line with your answer.

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

https://github.com/googleapis/google-cloud-cpp.git
5ad364581ed52ce457a2f79449964063e96242c6
5ad364581ed52ce457a2f79449964063e96242c6

Have you found anything relevant by searching the web?

Any other information, logs, or outputs that you want to share?

[johnedmonds]

git bisect seems to show ce65de4 as the culprit.

[johnedmonds]

Disabling the flag (--experimental_generated_protos_in_virtual_imports=false) does not seem to fix it.

[lberki]

This is fixed grpc/grpc#19860 . I'll ping that pull request, although you'll also need to integrate it.

Alternatively, I could craft a patch like I do in envoyproxy/envoy#7850 . Which one do you prefer?

[johnedmonds]

Would we have to bump the grpc version in google-cloud-cpp?

[lberki]

Yep.

[lberki]

Clarification: you'd have to bump the grpc version number if you want to avoid patching grpc. If that's okay, I can send you a pull request right away.

[johnedmonds]

Thanks for the explanation! Just to clarify more, would we also need to bump the GRPC version in any Bazel projects using grpc (e.g. including internal projects)?

And I think bumping the GRPC version is fine. Though I'm curious if google-cloud-cpp is okay with us pulling in a specific commit or if we need to wait for a release of GRPC.

[lberki]

Yep, or use the patch above. Unfortunately, I haven't heard back from the gRPC folks yet so I don't know what next :(

[johnedmonds]

Good news: I've tested bumping the grpc version used by google-cloud-cpp and now it successfully compiles with bazel 0.29.0rc6. I've got the google-cloud-cpp changes required for the version bump internally and I can work on open sourcing them. We can probably mark this bug as closed.

[lberki]

Closing then. Apologies for the breakage. The good news is that protobuf support built into Bazel is almost out of the woods now: all I'll need to do is to remove [ProtoInfo].check_deps_sources and then no more breaking changes are expected for a while.