Difficulty referencing an external proto on linux. #22

pcj · 2016-10-04T21:18:08Z

Discussed at https://groups.google.com/d/msg/bazel-discuss/TyI0mMX_fQo/0audY7sTAQAJ with @jhahn21.

Another thing to try is to define a java_proto_library for the wkt_protos and depend on that, such as:

package(default_visibility = ["//visibility:public"])

load("@org_pubref_rules_protobuf//java:rules.bzl", "java_proto_library")

java_proto_library(
    name = "wkt_protos",
    protos = ["@com_github_google_protobuf//:well_known_protos"],
    imports = ['external/com_github_google_protobuf/src'],
)

java_proto_library(
    name = "java",
    protos = ["common.proto"],
    proto_deps = [
        "wkt_protos",
    ],
    imports = ['external/com_github_google_protobuf/src'],
    inputs = ["@com_github_google_protobuf//:well_known_protos"],
)

junghoahnsc · 2016-10-04T21:25:33Z

Just tried:

ERROR: /home/jungho.ahn/work/p/rules_protobuf_java_test/examples/proto/BUILD:19:1: error executing shell command: 'bazel-out/host/bin/external/com_github_google_protobuf/protoc --descriptor_set_out=bazel-out/local-fastbuild/genfiles/examples/proto/java.pb.descriptor_set --java_out=bazel-out/local-fastbuild/genf...' failed: linux-sandbox failed: error executing command
  (cd /usr/local/share/users/jungho.ahn/cache/bazel/_bazel_jungho.ahn/7d1ea5683c20646cb1228e609b45731f/execroot/rules_protobuf_java_test && \
  exec env - \
  /usr/local/share/users/jungho.ahn/cache/bazel/_bazel_jungho.ahn/7d1ea5683c20646cb1228e609b45731f/execroot/rules_protobuf_java_test/_bin/linux-sandbox @/usr/local/share/users/jungho.ahn/cache/bazel/_bazel_jungho.ahn/7d1ea5683c20646cb1228e609b45731f/execroot/rules_protobuf_java_test/bazel-sandbox/18414400-34a3-459f-89d0-39d3fd5e3623-1.params -- /bin/bash -c 'bazel-out/host/bin/external/com_github_google_protobuf/protoc --descriptor_set_out=bazel-out/local-fastbuild/genfiles/examples/proto/java.pb.descriptor_set --java_out=bazel-out/local-fastbuild/genfiles/examples/proto/java.pb_java.jar --plugin=protoc-gen-grpc-java=bazel-out/host/genfiles/external/org_pubref_rules_protobuf/third_party/protoc_gen_grpc_java/protoc_gen_grpc_java --grpc-java_out=bazel-out/local-fastbuild/genfiles/examples/proto/java.pb_java.jar --proto_path=external/com_github_google_protobuf/src --proto_path=. examples/proto/common.proto').
external/com_github_google_protobuf/src: warning: directory does not exist.
google/protobuf/descriptor.proto: File not found.
examples/proto/common.proto: Import "google/protobuf/descriptor.proto" was not found or had errors.
examples/proto/common.proto:12:3: "google.protobuf.FileDescriptorProto" is not defined.
Target //examples/proto:java failed to build

pcj · 2016-10-04T21:36:31Z

Can you try pcj/rules_protobuf_java_test@2fb34ae? Could run into the same issue.

junghoahnsc · 2016-10-04T21:47:08Z

Sure. Yes, I got the same issue:

ubuntu-trusty-1:~/.../rules_protobuf_java_test/examples/proto] bazel build :java
.
WARNING: /usr/local/share/users/jungho.ahn/cache/bazel/_bazel_jungho.ahn/7d1ea5683c20646cb1228e609b45731f/external/org_pubref_rules_protobuf/protobuf/internal/proto_compile.bzl:413:5: proto_compile examples/proto/BUILD:java.pb.
WARNING: /usr/local/share/users/jungho.ahn/cache/bazel/_bazel_jungho.ahn/7d1ea5683c20646cb1228e609b45731f/external/org_pubref_rules_protobuf/protobuf/internal/proto_compile.bzl:380:5:
************************************************************
cd $(bazel info execution_root) && bazel-out/host/bin/external/com_github_google_protobuf/protoc \
--descriptor_set_out=bazel-out/local-fastbuild/genfiles/examples/proto/java.pb.descriptor_set \
--java_out=bazel-out/local-fastbuild/genfiles/examples/proto/java.pb_java.jar \
--plugin=protoc-gen-grpc-java=bazel-out/host/genfiles/external/org_pubref_rules_protobuf/third_party/protoc_gen_grpc_java/protoc_gen_grpc_java \
--grpc-java_out=bazel-out/local-fastbuild/genfiles/examples/proto/java.pb_java.jar \
--proto_path=external/com_github_google_protobuf/src \
--proto_path=. \
examples/proto/common.proto
************************************************************
examples/proto/java.pb_java.jar
examples/proto/java.pb.descriptor_set
************************************************************
.
INFO: Found 1 target...
ERROR: /home/jungho.ahn/work/p/rules_protobuf_java_test/examples/proto/BUILD:27:1: error executing shell command: 'bazel-out/host/bin/external/com_github_google_protobuf/protoc --descriptor_set_out=bazel-out/local-fastbuild/genfiles/examples/proto/java.pb.descriptor_set --java_out=bazel-out/local-fastbuild/genf...' failed: linux-sandbox failed: error executing command /usr/local/share/users/jungho.ahn/cache/bazel/_bazel_jungho.ahn/7d1ea5683c20646cb1228e609b45731f/execroot/rules_protobuf_java_test/_bin/linux-sandbox ... (remaining 5 argument(s) skipped).
external/com_github_google_protobuf/src: warning: directory does not exist.
google/protobuf/descriptor.proto: File not found.
examples/proto/common.proto: Import "google/protobuf/descriptor.proto" was not found or had errors.
examples/proto/common.proto:12:3: "google.protobuf.FileDescriptorProto" is not defined.
Target //examples/proto:java failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 33.262s, Critical Path: 20.48s

BTW, I'm also trying to generate for Go and Python, but Go failed like:

$ bazel build --spawn_strategy=standalone :goproto
ERROR: /home/jungho.ahn/work/p/rules_protobuf_java_test/examples/proto/BUILD:46:1: null failed: goproto.a.GoCompileFile.params failed: error executing command bazel-out/local-fastbuild/bin/examples/proto/bazel-out/local-fastbuild/bin/examples/proto/goproto.a.GoCompileFile.params: com.google.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
bazel-out/local-fastbuild/genfiles/examples/proto/common.pb.go:19: can't find import: "google/protobuf"
Target //examples/proto:goproto failed to build

BUILD:

go_proto_library(
    name = "goproto",
    protos = ["common.proto"],
    imports = ['external/com_github_google_protobuf/src'],
)

pcj · 2016-10-04T22:25:35Z

Assuming you called go_proto_repositories() in the WORKSPACE, looks like same issue.

junghoahnsc · 2016-10-04T22:27:04Z

Yes I called go_proto_repositories().
But with Go, it failed even when I gave --spawn_strategy=standalone.

junghoahnsc · 2016-10-05T00:05:45Z

Looks like there are two issues.

I tried google/protobuf/timestamp.proto instead of descriptor.prot.
It seems to work with deps= [ "@com_github_golang_protobuf//:ptypes/timestamp"],
but it failed with deps= [ "@com_github_golang_protobuf//:ptypes"]

ERROR: /usr/local/share/users/jungho.ahn/cache/bazel/_bazel_jungho.ahn/7d1ea5683c20646cb1228e609b45731f/external/com_github_golang_protobuf/BUILD:76:1: in go_library rule @com_github_golang_protobuf//:ptypes:
Traceback (most recent call last):
        File "/usr/local/share/users/jungho.ahn/cache/bazel/_bazel_jungho.ahn/7d1ea5683c20646cb1228e609b45731f/external/com_github_golang_protobuf/BUILD", line 76
            go_library(name = 'ptypes')
        File "/usr/local/share/users/jungho.ahn/cache/bazel/_bazel_jungho.ahn/7d1ea5683c20646cb1228e609b45731f/external/io_bazel_rules_go/go/def.bzl", line 275, in go_library_impl
            fail("may not be empty", "srcs")
attribute srcs: may not be empty.
ERROR: Analysis of target '//examples/proto:proto_go' failed; build aborted.

Unlike other google/protobuf/*, descriptor.proto has a different go_package.
https://github.com/google/protobuf/blob/master/src/google/protobuf/descriptor.proto#L43

I think M param should be passed to --go_out like Mgoogle/protobuf/descriptor.proto=github.com/golang/protobuf/protoc-gen-go/descriptor

and deps= [ "@com_github_golang_protobuf//:protoc-gen-go/descriptor" ] should be added.

junghoahnsc · 2016-10-05T00:22:19Z

Looks like it compiled with
proto_compile_args = { 'importmap': {'google/protobuf/descriptor.proto': 'github.com/golang/protobuf/protoc-gen-go/descriptor'}},

Is this a right way to do that?

pcj · 2016-10-05T01:40:09Z

Yes! (nice job, BTW).

It should be easier than all this.

The importmap option should better exposed such as go_proto_library(importmap={...}).
This should go in the go/README.md documentation.
The importmapping in go is always a bit confusing. It would be nice to have a table that maps out the accessible protos that exist in the provided deps.

Do you agree? I'll make a PR to update 1.

junghoahnsc · 2016-10-05T01:51:08Z

Yeah. It would be better!

- Partially addresses issue #22.

- Partially addresses issue pubref#22.

sblaes · 2016-12-02T06:05:41Z

Did you have any success getting well known types to work with java_proto_library?

pcj · 2017-09-09T04:14:53Z

Closing due to inactivity.

pcj added a commit that referenced this issue Oct 5, 2016

Adds example of importing protobuf well-known-types in golang.

5694622

- Partially addresses issue #22.

pcj added a commit that referenced this issue Oct 5, 2016

Adds example of importing protobuf well-known-types in golang.

6b0e1ae

- Partially addresses issue #22.

achew22 pushed a commit to achew22/rules_protobuf that referenced this issue Oct 8, 2016

Adds example of importing protobuf well-known-types in golang.

f912ced

- Partially addresses issue pubref#22.

cheister mentioned this issue Apr 22, 2017

Automatically generate a protoc friendly dependency tree from the proto_deps #90

Closed

pcj closed this as completed Sep 9, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Difficulty referencing an external proto on linux. #22

Difficulty referencing an external proto on linux. #22

pcj commented Oct 4, 2016

junghoahnsc commented Oct 4, 2016

pcj commented Oct 4, 2016

junghoahnsc commented Oct 4, 2016

pcj commented Oct 4, 2016

junghoahnsc commented Oct 4, 2016

junghoahnsc commented Oct 5, 2016

junghoahnsc commented Oct 5, 2016

pcj commented Oct 5, 2016

junghoahnsc commented Oct 5, 2016

sblaes commented Dec 2, 2016

pcj commented Sep 9, 2017

Difficulty referencing an external proto on linux. #22

Difficulty referencing an external proto on linux. #22

Comments

pcj commented Oct 4, 2016

junghoahnsc commented Oct 4, 2016

pcj commented Oct 4, 2016

junghoahnsc commented Oct 4, 2016

pcj commented Oct 4, 2016

junghoahnsc commented Oct 4, 2016

junghoahnsc commented Oct 5, 2016

junghoahnsc commented Oct 5, 2016

pcj commented Oct 5, 2016

junghoahnsc commented Oct 5, 2016

sblaes commented Dec 2, 2016

pcj commented Sep 9, 2017