Skip to content

Commit

Permalink
Update protocol buffers blogpost for the deprecation of native Protob…
Browse files Browse the repository at this point in the history 
…uf rules (#191)

* Update protocol buffers blogpost for the deprecation of native Protobuf rules

* Fill-in missing stuff

* cgrushko/proto_library is updated

* Fix typo

* Recommend Protobuf 3.10 and update java lite

* Create 2019-10-01-protobuf-updates.md

* Update 2017-02-27-protocol-buffers.md

* Fix url
  • Loading branch information
@Yannic @aiuto
Yannic authored and aiuto committed Oct 28, 2019
1 parent a83017b commit 2183125
Show file tree
Hide file tree
Showing 2 changed files with 89 additions and 29 deletions.
108 changes: 79 additions & 29 deletions _posts/2017-02-27-protocol-buffers.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,10 @@ authors:
- cgrushko
---

Bazel currently provides built-in rules for Java, JavaLite and C++.
Bazel currently provides rules for Java, JavaLite and C++.

<!-- TODO(yannic): Create documentation for rules_proto and link to that instead.
https://github.com/bazelbuild/bazel/issues/9203 -->
[`proto_library`]({{ site.docs_site_url }}/be/protocol-buffer.html#proto_library)
is a language-agnostic rule that describes relations between `.proto` files.

Expand Down Expand Up @@ -35,34 +37,78 @@ The following satisfies these dependencies:

> TIP: Clone [https://github.com/cgrushko/proto_library](https://github.com/cgrushko/proto_library) to try protobufs in Bazel now.
> **Update (January 2019)**: If you're using Bazel 0.21.0 or later, the minimum Protocol Buffer version required is [**3.6.1.2**](https://github.com/protocolbuffers/protobuf/releases/tag/v3.6.1.2). See this [pull request](https://github.com/protocolbuffers/protobuf/pull/4650) for more information.
> **Update (August 2019)**: We strongly recommend to use
[Protocol Buffer version 3.10](https://github.com/protocolbuffers/protobuf/releases/tag/v3.10.0-rc1)
ar later to maximize compatibility with future Bazel releases.

```python
# proto_library, cc_proto_library, and java_proto_library rules implicitly
# depend on @com_google_protobuf for protoc and proto runtimes.
# This statement defines the @com_google_protobuf repo.
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

# rules_cc defines rules for generating C++ code from Protocol Buffers.
http_archive(
name = "rules_cc",
sha256 = "35f2fb4ea0b3e61ad64a369de284e4fbbdcdba71836a5555abb5e194cf119509",
strip_prefix = "rules_cc-624b5d59dfb45672d4239422fa1e3de1822ee110",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/rules_cc/archive/624b5d59dfb45672d4239422fa1e3de1822ee110.tar.gz",
"https://github.com/bazelbuild/rules_cc/archive/624b5d59dfb45672d4239422fa1e3de1822ee110.tar.gz",
],
)

# rules_java defines rules for generating Java code from Protocol Buffers.
http_archive(
name = "com_google_protobuf",
sha256 = "cef7f1b5a7c5fba672bec2a319246e8feba471f04dcebfe362d55930ee7c1c30",
strip_prefix = "protobuf-3.5.0",
urls = ["https://github.com/google/protobuf/archive/v3.5.0.zip"],
name = "rules_java",
sha256 = "ccf00372878d141f7d5568cedc4c42ad4811ba367ea3e26bc7c43445bbc52895",
strip_prefix = "rules_java-d7bf804c8731edd232cb061cb2a9fe003a85d8ee",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/rules_java/archive/d7bf804c8731edd232cb061cb2a9fe003a85d8ee.tar.gz",
"https://github.com/bazelbuild/rules_java/archive/d7bf804c8731edd232cb061cb2a9fe003a85d8ee.tar.gz",
],
)

# rules_proto defines abstract rules for building Protocol Buffers.
http_archive(
name = "rules_proto",
sha256 = "57001a3b33ec690a175cdf0698243431ef27233017b9bed23f96d44b9c98242f",
strip_prefix = "rules_proto-9cd4f8f1ede19d81c6d48910429fe96776e567b1",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/rules_proto/archive/9cd4f8f1ede19d81c6d48910429fe96776e567b1.tar.gz",
"https://github.com/bazelbuild/rules_proto/archive/9cd4f8f1ede19d81c6d48910429fe96776e567b1.tar.gz",
],
)

# java_lite_proto_library rules implicitly depend on @com_google_protobuf_javalite//:javalite_toolchain,
# which is the JavaLite proto runtime (base classes and common utilities).
http_archive(
name = "com_google_protobuf_javalite",
sha256 = "d8a2fed3708781196f92e1e7e7e713cf66804bd2944894401057214aff4f468e",
strip_prefix = "protobuf-5e8916e881c573c5d83980197a6f783c132d4276",
urls = ["https://github.com/google/protobuf/archive/5e8916e881c573c5d83980197a6f783c132d4276.zip"],
sha256 = "311b29b8d0803ab4f89be22ff365266abb6c48fd3483d59b04772a144d7a24a1",
strip_prefix = "protobuf-7b64714af67aa967dcf941df61fe5207975966be",
urls = [
"https://github.com/google/protobuf/archive/7b64714af67aa967dcf941df61fe5207975966be.zip",
],
)

load("@rules_cc//cc:repositories.bzl", "rules_cc_dependencies")
rules_cc_dependencies()

load("@rules_java//java:repositories.bzl", "rules_java_dependencies", "rules_java_toolchains")
rules_java_dependencies()
rules_java_toolchains()

load("@rules_proto//proto:repositories.bzl", "rules_proto_dependencies", "rules_proto_toolchains")
rules_proto_dependencies()
rules_proto_toolchains()
```

### BUILD files

> TIP: Clone [https://github.com/cgrushko/proto_library](https://github.com/cgrushko/proto_library) to try protobufs in Bazel now.
```python
load("@rules_cc//cc:defs.bzl", "cc_proto_library")
load("@rules_java//java:defs.bzl", "java_proto_library")
load("@rules_proto//proto:defs.bzl", "proto_library")

java_proto_library(
name = "person_java_proto",
deps = [":person_proto"],
Expand Down Expand Up @@ -118,32 +164,35 @@ files in a project.
1. One proto_library rule per `.proto` file.
2. A file named `foo.proto` will be in a rule named `foo_proto`, which is
located in the same package.
3. A `X_proto_library` that wraps a `proto_library` named `foo_proto` should be
called `foo_X_proto`, and be located in the same package.
3. A `<lang>_proto_library` that wraps a `proto_library` named `foo_proto` should be
called `foo_<lang>_proto`, and be located in the same package.

## FAQ

**Q:** I already have rules named `java_proto_library` and `cc_proto_library`.
Will there be a problem?<br />
**A:** No. Since Skylark extensions imported through `load` statements take
precedence over native rules with the same name, the new rule should not affect
existing usage of the `java_proto_library` macro.
**Q:** I already have rules or macros named `proto_library`, `java_proto_library`,
and `cc_proto_library`. Will there be a problem?<br />
**A:** No. Since these rules are explicitely loaded through `load` statements,
the new rule should not affect existing usage of the rules or macros.

<!-- TODO(yannic): This is outdated. -->
**Q:** How do I use gRPC with these rules?<br />
**A:** The Bazel rules do not generate RPC code since `protobuf` is independent
of any RPC system. We will work with the gRPC team to create Skylark extensions
to do so. ([C++ Issue](https://github.com/grpc/grpc/issues/9873), [Java
Issue](https://github.com/grpc/grpc-java/issues/2756))

**Q:** Do you plan to release additional languages?<br />
**A:** We can relatively easily create `py_proto_library`. Our end goal is to
improve Skylark to the point where these rules can be written in Skylark, making
them independent of Bazel.
**A:** We're currently [rewriting the existing](https://docs.google.com/document/d/1u95vlQ1lWeQNR4bUw5T4cMeHTGJla2_e1dHHx7v4Dvg/edit)
rules for C++ and Java in Starlark, making them independent of Bazel.
We may add rules for other languages after this work is finished.

**Q:** How does one use well-known types? (e.g., `any.proto`,
`descriptor.proto`)<br />
**A:** See [the example repo](https://github.com/cgrushko/proto_library), particularly https://github.com/cgrushko/proto_library/blob/beae7b78b85b3af51d3ea54663c421ebde97dc10/src/BUILD#L42.

<!-- TODO(yannic): This is outdated. -->
<!-- TODO(yannic): rules_proto will contain a framework for writing rules and a guide
how to do so. Link to that instead. -->
**Q:** Any tips for writing my own such rules?<br />
**A:** First, make sure you're able to register actions that compile your target
language. (as far as I know, Bazel Python actions are not exposed to Skylark,
Expand All @@ -162,14 +211,15 @@ good example.

### Implicit Dependencies and Proto Toolchains

<!-- TODO(yannic): This will need to be updated when the flags are removed. -->
The `proto_library` rule implicitly depends on `@com_google_protobuf//:protoc`,
which is the protocol buffer compiler. It must be a binary rule (in protobuf,
it's a `cc_binary`). The rule can be overridden using the `--proto_compiler`
command-line flag.

Most `X_proto_library` rules implicitly depend on
`@com_google_protobuf//:X_toolchain`, which is a `proto_lang_toolchain` rule.
These rules can be overridden using the `--proto_toolchain_for_X` command-line
Most `<lang>_proto_library` rules implicitly depend on
`@com_google_protobuf//:<lang>_toolchain`, which is a `proto_lang_toolchain` rule.
These rules can be overridden using the `--proto_toolchain_for_<lang>` command-line
flags.

A `proto_lang_toolchain` rule describes how to call the protocol compiler, and
Expand All @@ -179,18 +229,18 @@ repository](https://github.com/google/protobuf/blob/b4b0e304be5a68de3d0ee1af9b28

### Bazel Aspects

The `X_proto_library` rules are implemented using [Bazel
The `<lang>_proto_library` rules are implemented using [Bazel
Aspects]({{ site.docs_site_url }}/skylark/aspects.html) to have
the best of two worlds -

1. Only need a single `X_proto_library` rule for an arbitrarily-large proto
1. Only need a single `<lang>_proto_library` rule for an arbitrarily-large proto
graph.
2. Incrementality, caching and no linking errors.

Conceptually, an `X_proto_library` rule creates a shadow graph of the
Conceptually, an `<lang>_proto_library` rule creates a shadow graph of the
`proto_library` it depends on, and each shadow node calls protocol-compiler and
then compiles the generated code. This way, if there are multiple paths from a
rule to a `proto_library` through `X_proto_library`, they all share the same
rule to a `proto_library` through `<lang>_proto_library`, they all share the same
node.

### Descriptor Sets
Expand Down
10 changes: 10 additions & 0 deletions _posts/2019-10-01-protobuf-updates.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
---
layout: posts
title: "Update on using Protocol Buffers in Bazel"
authors:
- Yannic
---

We’re pleased to announce that we updated the instructions for using
[Protocol Buffers in Bazel](/2017/02/27/protocol-buffers.html)
to reflect recent and upcoming changes to Bazel.

0 comments on commit 2183125

Please sign in to comment.