Skip to content

Latest commit

 

History

History
503 lines (411 loc) · 25.5 KB

core.rst

File metadata and controls

503 lines (411 loc) · 25.5 KB
Raw

Go Protocol buffers

rules_go provides rules that generate Go packages from .proto files. These packages can be imported like regular Go libraries.

depth

2

Overview

Protocol buffers are built with the three rules below. go_proto_library and go_proto_compiler may be loaded from @io_bazel_rules_go//proto:def.bzl.

  • proto_library: This is a Bazel built-in rule. It lists a set of .proto files in its srcs attribute and lists other proto_library dependencies in its deps attribute. proto_library rules may be referenced by language-specific code generation rules like java_proto_library and go_proto_library.
  • go_proto_library: Generates Go code from .proto files using one or more proto plugins, then builds that code into a Go library. go_proto_library references proto_library sources via the proto attribute. They may reference other go_proto_library and go_library dependencies via the deps attributes. go_proto_library rules can be depended on or embedded directly by go_library and go_binary.
  • go_proto_compiler: Defines a protoc plugin. By default, go_proto_library generates Go code with the default Go plugin, but other plugins can be used by setting the compilers attribute. A few common plugins are provided in @io_bazel_rules_go//proto.

The go_proto_compiler rule produces a GoProtoCompiler provider. If you need a greater degree of customization (for example, if you don't want to use protoc), you can implement a compatible rule that returns one of these.

The go_proto_library rule produces the normal set of Go providers. This makes it compatible with other Go rules for use in deps and embed attributes.

Avoiding conflicts

When linking programs that depend on protos, care must be taken to ensure that the same proto isn't registered by more than one package. This may happen if you depend on a go_proto_library and a vendored go_library generated from the same .proto files. You may see compile-time, link-time, or run-time errors as a result of this.

There are two main ways to avoid conflicts.

Option 1: Use go_proto_library exclusively

You can avoid proto conflicts by using go_proto_library to generate code at build time and avoiding go_library rules based on pre-generated .pb.go files.

Gazelle generates rules in this mode by default. When .proto files are present, it will generate go_proto_library rules and go_library rules that embed them (which are safe to use). Gazelle will automatically exclude .pb.go files that correspond to .proto files. If you have .proto files belonging to multiple packages in the same directory, add the following directives to your root build file:

# gazelle:proto package
# gazelle:proto_group go_package

rules_go provides go_proto_library rules for commonly used proto libraries. The Well Known Types can be found in the @io_bazel_rules_go//proto/wkt package. There are implicit dependencies of go_proto_library rules that use the default compiler, so they don't need to be written explicitly in deps. You can also find rules for Google APIs and gRPC in @go_googleapis//. You can list these rules with the commands:

$ bazel query 'kind(go_proto_library, @io_bazel_rules_go//proto/wkt:all)'
$ bazel query 'kind(go_proto_library, @go_googleapis//...)'

Some commonly used Go libraries, such as github.com/golang/protobuf/ptypes, depend on the Well Known Types. In order to avoid conflicts when using these libraries, separate versions of these libraries are provided with go_proto_library dependencies. Gazelle resolves imports of these libraries automatically. For example, it will resolve ptypes as @com_github_golang_protobuf//ptypes:go_default_library_gen.

Option 2: Use pre-generated .pb.go files

You can also avoid conflicts by generating .pb.go files ahead of time and using those exclusively instead of using go_proto_library. This may be a better option for established Go projects that also need to build with go build.

Gazelle can generate rules for projects built in this mode. Add the following comment to your root build file:

# gazelle:proto disable_global

This prevents Gazelle from generating go_proto_library rules. .pb.go files won't be excluded, and all special cases for imports (such as ptypes) are disabled.

If you have go_repository rules in your WORKSPACE file that may have protos, you'll also need to add build_file_proto_mode = "disable_global" to those as well.

go_repository(
    name = "com_example_some_project",
    importpath = "example.com/some/project",
    tag = "v0.1.2",
    build_file_proto_mode = "disable_global",
)

A note on vendored .proto files

Bazel can only handle imports in .proto files that are relative to a repository root directory. This means, for example, if you import "foo/bar/baz.proto", that file must be in the directory foo/bar, not vendor/example.com/repo/foo/bar.

If you have proto files that don't conform to this convention, follow the instructions for using pre-generated .pb.go files above.

The Bazel tracking issue for supporting this is bazelbuild/bazel#3867.

API

go_proto_library

go_proto_library generates a set of .go files from a set of .proto files (specified in a proto_library rule), then builds a Go library from those files. go_proto_library can be imported like any go_library rule.

Providers

Attributes

Name Type Default value
name string mandatory value

A unique name for thi

By convention, and in should be a name like or the last component proto_library ref

s rule.

order to interoperate foo_go_proto, wher of the proto package n

erenced by proto sh

 cleanly with Gazelle, this e foo is the Go package name ame (hopefully the same). The ould be named foo_proto.
proto label mandatory value
Points to the ``proto should generate code _library`` containing t from. Avoid using this he .proto sources this rule argument, use protos instead.
protos label mandatory value
List of ``proto_libra code from. This argum ry`` targets containing ent should be used inst

the .proto sources this rule should generate

ead of proto argument.
deps label_list []

List of Go libraries a list of go_proto_proto_library`` rul

Additional dependenci default compiler impl rules for the Well Kn

this library depends on libraryrules that co e referenced byproto

es may be added by the icitly adds dependencie own Types.

directly. Usually, this will be

rrespond to the deps of the . proto compiler. For example, the s on thego_proto_library``
importpath string mandatory value

The source import pat library using this pa inherited from one of

importpath must m option go_package for protos importing

h of this library. Othe th. This must be specif the targets in ``embed

atch the import path sp . The option determines this proto will import

 r libraries can import this ied in go_proto_library or . ecified in.protofiles using how.pb.go`` files generated this package.
importmap string ""
The Go package path o compiler and linker, set to prevent a bina path, e.g., from diff f this library. This is but it may also be seen ry from linking multipl erent vendor directorie

mostly only visible to the in stack traces. This may be

e packages with the same import s.
embed label_list []
List of Go libraries and deps from the is compiled. Embedded Go package name. that should be combined se libraries will be in libraries must have th

with this library. The srcs

corporated into this library when it e same importpath and
gc_goopts string_list []
List of flags to add compiler. Subject to to the Go compilation c `Make variable substitu ommand when using the gc tion`_ and Bourne shell tokenization.
compiler label None
Equivalent to ``compi lers`` with a single la bel.
compilers label_list ["@io_bazel_rules_go//proto:go_proto"]
List of rules produci go_proto_compiler protoc plugins used t some options. ng GoProtoCompiler p rules). This is usually o generate Go code. See
roviders (normally

understood to be a list of Predefined plugins for

Example: Basic proto

Suppose you have two .proto files in separate packages: foo/foo.proto and bar/bar.proto. foo/foo.proto looks like this:

syntax = "proto3";

option go_package = "example.com/repo/foo";

import "google/protobuf/any.proto";
import "bar/bar.proto";

message Foo {
  bar.Bar x = 1;
  google.protobuf.Any y = 2;
};

In foo/BUILD.bazel, we need to declare a proto_library rule that lists foo.proto in its srcs attribute. Since we import some other protos, we also need a label in deps for each imported package. We will need to create another proto_library in bar/BUILD.bazel, but we can use an existing library for any.proto, since it's one of the Well Known Types.

proto_library(
    name = "foo_proto",
    srcs = ["foo.proto"],
    deps = [
        "//bar:bar_proto",
        "@com_google_protobuf//:any_proto",
    ],
    visibility = ["//visibility:public"],
)

In order to these this proto in Go, we need to declare a go_proto_library that references to proto_library to be built via the proto attribute. Like go_library, an importpath attribute needs to be declared. Ideally, this should match the option go_package declaration in the .proto file, but this is not required. We also need to list Go packages that the generated Go code imports in the deps attributes. Generally, deps in go_proto_library will correspond with deps in proto_library, but the Well Known Types don't need to be listed (they are added automatically by the compiler in use).

load("@io_bazel_rules_go//proto:def.bzl", "go_proto_library")

go_proto_library(
    name = "foo_go_proto",
    importpath = "example.com/repo/foo",
    proto = ":foo_proto",
    visibility = ["//visibility:public"],
    deps = ["//bar:bar_go_proto"],
)

This library can be imported like a regular Go library by other rules.

load("@io_bazel_rules_go//go:def.bzl", "go_binary")

go_binary(
    name = "main",
    srcs = ["main.go"],
    deps = ["//foo:foo_go_proto"],
)

If you need to add additional source files to a package built from protos, you can do so with a separate go_library that embeds the go_proto_library.

load("@io_bazel_rules_go//go:def.bzl", "go_library")

go_library(
    name = "foo",
    srcs = ["extra.go"],
    embed = [":foo_go_proto"],
    importpath = "example.com/repo/foo",
    visibility = ["//visibility:public"],
)

For convenience, proto_library, go_proto_library, and go_binary can all be generated by Gazelle.

Example: gRPC

To compile protos that contain service definitions, just use the go_grpc plugin.

load("@io_bazel_rules_go//proto:def.bzl", "go_proto_library")

proto_library(
    name = "foo_proto",
    srcs = ["foo.proto"],
    visibility = ["//visibility:public"],
)

go_proto_library(
    name = "foo_go_proto",
    compilers = ["@io_bazel_rules_go//proto:go_grpc"],
    importpath = "example.com/repo/foo",
    proto = ":foo_proto",
    visibility = ["//visibility:public"],
    deps = ["//bar:bar_go_proto"],
)

go_proto_compiler

go_proto_compiler describes a plugin for protoc, the proto compiler. Different plugins will generate different Go code from the same protos. Compilers may be chosen through the compilers attribute of go_proto_library.

Several instances of this rule are listed in Predefined plugins. You will only need to use this rule directly if you need a plugin which is not there.

Providers

Attributes

Name Type Default value
name string mandatory value
A unique name for this rule.
deps label_list []
List of Go libraries that Go implicitly. Rules in this lis should contain libraries for code generated by thi t must produce the `GoL the Well Known Types at s compiler depends on ibrary`_ provider. This least.
options string_list []
List of command line options be preceded by --option. to be passed to the com piler. Each option will
suffix string .pb.go
File name suffix of generated one Go file will be generated will have the .proto suffix r foo.proto will become ``f

Go files. ``go_proto_c for each input .proto

emoved and this suffix oo.pb.go``.		 ompiler`` assumes that file. Output file names appended. For example,
valid_archive bool True
Whether code generated by thi archive file without addition s compiler can be compi al sources. led into a standalone
import_path_option bool True
When true, the importpath using this compiler will be p --option import_path={}.

attribute from ``go_pr

assed to the compiler o		 oto_library`` rules n the command line as
plugin label @com_github_golang_protobuf//protoc-gen-go
The plugin to use with protoc produce an executable file. via the --plugin o ption. This rule must

Predefined plugins

Several go_proto_compiler rules are predefined in @io_bazel_rules_go//proto.

  • go_proto: default plugin from github.com/golang/protobuf.
  • go_grpc: default gRPC plugin.
  • go_proto_validate: validator plugin from github.com/mwitkow/go-proto-validators. Generates Validate methods.
  • gogoprotobuf plugins for the variants combo, gofast, gogo, gogofast, gogofaster, gogoslick, gogotypes, gostring. For each variant, there is a regular version (e.g., gogo_proto) and a gRPC version (e.g., gogo_grpc).

Providers

Providers are objects produced by Bazel rules and consumed by other rules that depend on them. See Go providers for information about Go providers, specifically GoLibrary, GoSource, and GoArchive.

GoProtoCompiler

GoProtoCompiler is the provider returned by the go_proto_compiler rule and anything compatible with it. The go_proto_library rule expects any rule listed in its compilers attribute to provide GoProtoCompiler. If the go_proto_compiler rule doesn't do what you need (e.g., you don't want to use protoc), you can write a new rule that produces this.

GoProtoCompiler is loaded from @io_bazel_rules_go//proto:def.bzl.

GoProtoCompiler has the fields described below. Additional fields may be added to pass information to the compile function. This interface is not final and may change in the future.

Name Type
deps :type:Target list
A list of Go libraries to be go_proto_library compiled GoLibrary, GoSource, and Go for the Well Known Types and
added as dependencies to any

with this compiler. Each target must provide

Archive. This list should include libraries anything else considered "standard".
compile Function
A function which declares out compiler.bzl for details. put files and actions when called. See
valid_archive bool
Whether the compiler produces methods to structs produced b

a complete Go library. Compilers that just add

y other compilers will set this to false.

Dependencies

In order to support protocol buffers, rules_go declares the external repositories listed below in go_rules_dependencies(). These repositories will only be downloaded if proto rules are used.

  • @com_google_protobuf (github.com/google/protobuf): Well Known Types and general proto support.
  • @com_github_golang_protobuf (github.com/golang/protobuf): standard Go proto plugin.
  • @com_github_mwitkow_go_proto_validators (github.com/mwitkow/go-proto-validators): validator plugin.
  • @com_github_gogo_protobuf (github.com/gogo/protobuf): gogoprotobuf plugins.
  • @org_golang_google_grpc (github.com/grpc/grpc-go: gRPC support.
  • gRPC dependencies
    • @org_golang_x_net (golang.org/x/net)
    • @org_golang_x_text (golang.org/x/text)
    • @org_golang_google_genproto (google.golang.org/genproto)