Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Moved proto deps into toolchain to avoid confusing boiler plate code #678

Merged
merged 5 commits into from
Apr 9, 2021
Merged

Moved proto deps into toolchain to avoid confusing boiler plate code #678

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
e43d96d
Moved protobuf and grpc deps into toolchain.
UebelAndre Apr 7, 2021
4948156
Regenerate documentation
UebelAndre Apr 7, 2021
492941b
Merge branch 'main' into proto
UebelAndre Apr 8, 2021
db14bb1
Merge branch 'main' into proto
UebelAndre Apr 9, 2021
8a0da94
Merge branch 'main' into proto
UebelAndre Apr 9, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 15 additions & 8 deletions docs/flatten.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -505,7 +505,6 @@ Example:

```python
load("//proto:proto.bzl", "rust_grpc_library")
load("//proto:toolchain.bzl", "GRPC_COMPILE_DEPS")

proto_library(
name = "my_proto",
Expand All @@ -520,7 +519,7 @@ rust_grpc_library(
rust_binary(
name = "my_service",
srcs = ["my_service.rs"],
deps = [":rust"] + GRPC_COMPILE_DEPS,
deps = [":rust"],
)
```

Expand All @@ -532,7 +531,7 @@ rust_binary(
| :------------- | :------------- | :------------- | :------------- | :------------- |
| <a id="rust_grpc_library-name"></a>name | A unique name for this target. | <a href="https://bazel.build/docs/build-ref.html#name">Name</a> | required | |
| <a id="rust_grpc_library-deps"></a>deps | List of proto_library dependencies that will be built. One crate for each proto_library will be created with the corresponding gRPC stubs. | <a href="https://bazel.build/docs/build-ref.html#labels">List of labels</a> | required | |
| <a id="rust_grpc_library-rust_deps"></a>rust_deps | The crates the generated library depends on. | <a href="https://bazel.build/docs/build-ref.html#labels">List of labels</a> | optional | [Label("//proto/raze:protobuf"), Label("//proto/raze:grpc"), Label("//proto/raze:tls_api"), Label("//proto/raze:tls_api_stub")] |
| <a id="rust_grpc_library-rust_deps"></a>rust_deps | The crates the generated library depends on. | <a href="https://bazel.build/docs/build-ref.html#labels">List of labels</a> | optional | [] |


<a id="#rust_library"></a>
Expand Down Expand Up @@ -682,7 +681,6 @@ Example:

```python
load("@rules_rust//proto:proto.bzl", "rust_proto_library")
load("@rules_rust//proto:toolchain.bzl", "PROTO_COMPILE_DEPS")

proto_library(
name = "my_proto",
Expand All @@ -697,7 +695,7 @@ proto_rust_library(
rust_binary(
name = "my_proto_binary",
srcs = ["my_proto_binary.rs"],
deps = [":rust"] + PROTO_COMPILE_DEPS,
deps = [":rust"],
)
```

Expand All @@ -709,15 +707,16 @@ rust_binary(
| :------------- | :------------- | :------------- | :------------- | :------------- |
| <a id="rust_proto_library-name"></a>name | A unique name for this target. | <a href="https://bazel.build/docs/build-ref.html#name">Name</a> | required | |
| <a id="rust_proto_library-deps"></a>deps | List of proto_library dependencies that will be built. One crate for each proto_library will be created with the corresponding stubs. | <a href="https://bazel.build/docs/build-ref.html#labels">List of labels</a> | required | |
| <a id="rust_proto_library-rust_deps"></a>rust_deps | The crates the generated library depends on. | <a href="https://bazel.build/docs/build-ref.html#labels">List of labels</a> | optional | [Label("//proto/raze:protobuf")] |
| <a id="rust_proto_library-rust_deps"></a>rust_deps | The crates the generated library depends on. | <a href="https://bazel.build/docs/build-ref.html#labels">List of labels</a> | optional | [] |


<a id="#rust_proto_toolchain"></a>

## rust_proto_toolchain

<pre>
rust_proto_toolchain(<a href="#rust_proto_toolchain-name">name</a>, <a href="#rust_proto_toolchain-edition">edition</a>, <a href="#rust_proto_toolchain-grpc_plugin">grpc_plugin</a>, <a href="#rust_proto_toolchain-proto_plugin">proto_plugin</a>, <a href="#rust_proto_toolchain-protoc">protoc</a>)
rust_proto_toolchain(<a href="#rust_proto_toolchain-name">name</a>, <a href="#rust_proto_toolchain-edition">edition</a>, <a href="#rust_proto_toolchain-grpc_compile_deps">grpc_compile_deps</a>, <a href="#rust_proto_toolchain-grpc_plugin">grpc_plugin</a>, <a href="#rust_proto_toolchain-proto_compile_deps">proto_compile_deps</a>,
<a href="#rust_proto_toolchain-proto_plugin">proto_plugin</a>, <a href="#rust_proto_toolchain-protoc">protoc</a>)
</pre>

Declares a Rust Proto toolchain for use.
Expand Down Expand Up @@ -761,7 +760,9 @@ See @rules_rust//proto:BUILD for examples of defining the toolchain.
| :------------- | :------------- | :------------- | :------------- | :------------- |
| <a id="rust_proto_toolchain-name"></a>name | A unique name for this target. | <a href="https://bazel.build/docs/build-ref.html#name">Name</a> | required | |
| <a id="rust_proto_toolchain-edition"></a>edition | The edition used by the generated rust source. | String | optional | "2015" |
| <a id="rust_proto_toolchain-grpc_compile_deps"></a>grpc_compile_deps | The crates the generated grpc libraries depends on. | <a href="https://bazel.build/docs/build-ref.html#labels">List of labels</a> | optional | [Label("//proto/raze:protobuf"), Label("//proto/raze:grpc"), Label("//proto/raze:tls_api"), Label("//proto/raze:tls_api_stub")] |
| <a id="rust_proto_toolchain-grpc_plugin"></a>grpc_plugin | The location of the Rust protobuf compiler plugin to generate rust gRPC stubs. | <a href="https://bazel.build/docs/build-ref.html#labels">Label</a> | optional | //proto:protoc_gen_rust_grpc |
| <a id="rust_proto_toolchain-proto_compile_deps"></a>proto_compile_deps | The crates the generated protobuf libraries depends on. | <a href="https://bazel.build/docs/build-ref.html#labels">List of labels</a> | optional | [Label("//proto/raze:protobuf")] |
| <a id="rust_proto_toolchain-proto_plugin"></a>proto_plugin | The location of the Rust protobuf compiler plugin used to generate rust sources. | <a href="https://bazel.build/docs/build-ref.html#labels">Label</a> | optional | //proto:protoc_gen_rust |
| <a id="rust_proto_toolchain-protoc"></a>protoc | The location of the <code>protoc</code> binary. It should be an executable target. | <a href="https://bazel.build/docs/build-ref.html#labels">Label</a> | optional | @com_google_protobuf//:protoc |

Expand Down Expand Up @@ -1362,11 +1363,17 @@ Declare dependencies needed for bindgen.
## rust_proto_repositories

<pre>
rust_proto_repositories()
rust_proto_repositories(<a href="#rust_proto_repositories-register_default_toolchain">register_default_toolchain</a>)
</pre>

Declare dependencies needed for proto compilation.

**PARAMETERS**


| Name | Description | Default Value |
| :------------- | :------------- | :------------- |
| <a id="rust_proto_repositories-register_default_toolchain"></a>register_default_toolchain | If True, the default [rust_proto_toolchain](#rust_proto_toolchain) (<code>@rules_rust//proto:default-proto-toolchain</code>) is registered. This toolchain requires a set of dependencies that were generated using [cargo raze](https://github.com/google/cargo-raze). These will also be loaded. | <code>True</code> |


<a id="#rust_repositories"></a>
Expand Down
23 changes: 15 additions & 8 deletions docs/rust_proto.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ Example:

```python
load("//proto:proto.bzl", "rust_grpc_library")
load("//proto:toolchain.bzl", "GRPC_COMPILE_DEPS")

proto_library(
name = "my_proto",
Expand All @@ -33,7 +32,7 @@ rust_grpc_library(
rust_binary(
name = "my_service",
srcs = ["my_service.rs"],
deps = [":rust"] + GRPC_COMPILE_DEPS,
deps = [":rust"],
)
```

Expand All @@ -45,7 +44,7 @@ rust_binary(
| :------------- | :------------- | :------------- | :------------- | :------------- |
| <a id="rust_grpc_library-name"></a>name | A unique name for this target. | <a href="https://bazel.build/docs/build-ref.html#name">Name</a> | required | |
| <a id="rust_grpc_library-deps"></a>deps | List of proto_library dependencies that will be built. One crate for each proto_library will be created with the corresponding gRPC stubs. | <a href="https://bazel.build/docs/build-ref.html#labels">List of labels</a> | required | |
| <a id="rust_grpc_library-rust_deps"></a>rust_deps | The crates the generated library depends on. | <a href="https://bazel.build/docs/build-ref.html#labels">List of labels</a> | optional | [Label("//proto/raze:protobuf"), Label("//proto/raze:grpc"), Label("//proto/raze:tls_api"), Label("//proto/raze:tls_api_stub")] |
| <a id="rust_grpc_library-rust_deps"></a>rust_deps | The crates the generated library depends on. | <a href="https://bazel.build/docs/build-ref.html#labels">List of labels</a> | optional | [] |


<a id="#rust_proto_library"></a>
Expand All @@ -62,7 +61,6 @@ Example:

```python
load("@rules_rust//proto:proto.bzl", "rust_proto_library")
load("@rules_rust//proto:toolchain.bzl", "PROTO_COMPILE_DEPS")

proto_library(
name = "my_proto",
Expand All @@ -77,7 +75,7 @@ proto_rust_library(
rust_binary(
name = "my_proto_binary",
srcs = ["my_proto_binary.rs"],
deps = [":rust"] + PROTO_COMPILE_DEPS,
deps = [":rust"],
)
```

Expand All @@ -89,15 +87,16 @@ rust_binary(
| :------------- | :------------- | :------------- | :------------- | :------------- |
| <a id="rust_proto_library-name"></a>name | A unique name for this target. | <a href="https://bazel.build/docs/build-ref.html#name">Name</a> | required | |
| <a id="rust_proto_library-deps"></a>deps | List of proto_library dependencies that will be built. One crate for each proto_library will be created with the corresponding stubs. | <a href="https://bazel.build/docs/build-ref.html#labels">List of labels</a> | required | |
| <a id="rust_proto_library-rust_deps"></a>rust_deps | The crates the generated library depends on. | <a href="https://bazel.build/docs/build-ref.html#labels">List of labels</a> | optional | [Label("//proto/raze:protobuf")] |
| <a id="rust_proto_library-rust_deps"></a>rust_deps | The crates the generated library depends on. | <a href="https://bazel.build/docs/build-ref.html#labels">List of labels</a> | optional | [] |


<a id="#rust_proto_toolchain"></a>

## rust_proto_toolchain

<pre>
rust_proto_toolchain(<a href="#rust_proto_toolchain-name">name</a>, <a href="#rust_proto_toolchain-edition">edition</a>, <a href="#rust_proto_toolchain-grpc_plugin">grpc_plugin</a>, <a href="#rust_proto_toolchain-proto_plugin">proto_plugin</a>, <a href="#rust_proto_toolchain-protoc">protoc</a>)
rust_proto_toolchain(<a href="#rust_proto_toolchain-name">name</a>, <a href="#rust_proto_toolchain-edition">edition</a>, <a href="#rust_proto_toolchain-grpc_compile_deps">grpc_compile_deps</a>, <a href="#rust_proto_toolchain-grpc_plugin">grpc_plugin</a>, <a href="#rust_proto_toolchain-proto_compile_deps">proto_compile_deps</a>,
<a href="#rust_proto_toolchain-proto_plugin">proto_plugin</a>, <a href="#rust_proto_toolchain-protoc">protoc</a>)
</pre>

Declares a Rust Proto toolchain for use.
Expand Down Expand Up @@ -141,7 +140,9 @@ See @rules_rust//proto:BUILD for examples of defining the toolchain.
| :------------- | :------------- | :------------- | :------------- | :------------- |
| <a id="rust_proto_toolchain-name"></a>name | A unique name for this target. | <a href="https://bazel.build/docs/build-ref.html#name">Name</a> | required | |
| <a id="rust_proto_toolchain-edition"></a>edition | The edition used by the generated rust source. | String | optional | "2015" |
| <a id="rust_proto_toolchain-grpc_compile_deps"></a>grpc_compile_deps | The crates the generated grpc libraries depends on. | <a href="https://bazel.build/docs/build-ref.html#labels">List of labels</a> | optional | [Label("//proto/raze:protobuf"), Label("//proto/raze:grpc"), Label("//proto/raze:tls_api"), Label("//proto/raze:tls_api_stub")] |
| <a id="rust_proto_toolchain-grpc_plugin"></a>grpc_plugin | The location of the Rust protobuf compiler plugin to generate rust gRPC stubs. | <a href="https://bazel.build/docs/build-ref.html#labels">Label</a> | optional | //proto:protoc_gen_rust_grpc |
| <a id="rust_proto_toolchain-proto_compile_deps"></a>proto_compile_deps | The crates the generated protobuf libraries depends on. | <a href="https://bazel.build/docs/build-ref.html#labels">List of labels</a> | optional | [Label("//proto/raze:protobuf")] |
| <a id="rust_proto_toolchain-proto_plugin"></a>proto_plugin | The location of the Rust protobuf compiler plugin used to generate rust sources. | <a href="https://bazel.build/docs/build-ref.html#labels">Label</a> | optional | //proto:protoc_gen_rust |
| <a id="rust_proto_toolchain-protoc"></a>protoc | The location of the <code>protoc</code> binary. It should be an executable target. | <a href="https://bazel.build/docs/build-ref.html#labels">Label</a> | optional | @com_google_protobuf//:protoc |

Expand All @@ -151,10 +152,16 @@ See @rules_rust//proto:BUILD for examples of defining the toolchain.
## rust_proto_repositories

<pre>
rust_proto_repositories()
rust_proto_repositories(<a href="#rust_proto_repositories-register_default_toolchain">register_default_toolchain</a>)
</pre>

Declare dependencies needed for proto compilation.

**PARAMETERS**


| Name | Description | Default Value |
| :------------- | :------------- | :------------- |
| <a id="rust_proto_repositories-register_default_toolchain"></a>register_default_toolchain | If True, the default [rust_proto_toolchain](#rust_proto_toolchain) (<code>@rules_rust//proto:default-proto-toolchain</code>) is registered. This toolchain requires a set of dependencies that were generated using [cargo raze](https://github.com/google/cargo-raze). These will also be loaded. | <code>True</code> |


23 changes: 14 additions & 9 deletions proto/proto.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,8 +35,6 @@ rust_proto_repositories()
load("@rules_proto//proto:defs.bzl", "ProtoInfo")
load(
"//proto:toolchain.bzl",
"GRPC_COMPILE_DEPS",
"PROTO_COMPILE_DEPS",
_generate_proto = "rust_generate_proto",
_generated_file_stem = "generated_file_stem",
)
Expand Down Expand Up @@ -214,6 +212,12 @@ def _rust_proto_compile(protos, descriptor_sets, imports, crate_name, ctx, is_gr
output_hash,
))

# Gather all dependencies for compilation
compile_action_deps = depset(
compile_deps +
proto_toolchain.grpc_compile_deps if is_grpc else proto_toolchain.proto_compile_deps,
)

return rustc_compile_action(
ctx = ctx,
toolchain = find_toolchain(ctx),
Expand All @@ -223,7 +227,7 @@ def _rust_proto_compile(protos, descriptor_sets, imports, crate_name, ctx, is_gr
type = "rlib",
root = lib_rs,
srcs = depset(srcs),
deps = depset(compile_deps),
deps = compile_action_deps,
proc_macro_deps = depset([]),
aliases = {},
output = rust_lib,
Expand Down Expand Up @@ -286,7 +290,6 @@ rust_proto_library = rule(
),
"rust_deps": attr.label_list(
doc = "The crates the generated library depends on.",
default = PROTO_COMPILE_DEPS,
),
"_cc_toolchain": attr.label(
default = Label("@bazel_tools//tools/cpp:current_cc_toolchain"),
Expand All @@ -310,6 +313,8 @@ rust_proto_library = rule(
str(Label("//rust:toolchain")),
"@bazel_tools//tools/cpp:toolchain_type",
],
# TODO: Remove once (bazelbuild/bazel#11584) is closed and the rules use
# the version of Bazel that issue was closed on as the min supported version
incompatible_use_toolchain_transition = True,
doc = """\
Builds a Rust library crate from a set of `proto_library`s.
Expand All @@ -318,7 +323,6 @@ Example:

```python
load("@rules_rust//proto:proto.bzl", "rust_proto_library")
load("@rules_rust//proto:toolchain.bzl", "PROTO_COMPILE_DEPS")

proto_library(
name = "my_proto",
Expand All @@ -333,7 +337,7 @@ proto_rust_library(
rust_binary(
name = "my_proto_binary",
srcs = ["my_proto_binary.rs"],
deps = [":rust"] + PROTO_COMPILE_DEPS,
deps = [":rust"],
)
```
""",
Expand Down Expand Up @@ -364,7 +368,6 @@ rust_grpc_library = rule(
),
"rust_deps": attr.label_list(
doc = "The crates the generated library depends on.",
default = GRPC_COMPILE_DEPS,
),
"_cc_toolchain": attr.label(
default = "@bazel_tools//tools/cpp:current_cc_toolchain",
Expand All @@ -388,14 +391,16 @@ rust_grpc_library = rule(
str(Label("//rust:toolchain")),
"@bazel_tools//tools/cpp:toolchain_type",
],
# TODO: Remove once (bazelbuild/bazel#11584) is closed and the rules use
# the version of Bazel that issue was closed on as the min supported version
incompatible_use_toolchain_transition = True,
doc = """\
Builds a Rust library crate from a set of `proto_library`s suitable for gRPC.

Example:

```python
load("//proto:proto.bzl", "rust_grpc_library")
load("//proto:toolchain.bzl", "GRPC_COMPILE_DEPS")

proto_library(
name = "my_proto",
Expand All @@ -410,7 +415,7 @@ rust_grpc_library(
rust_binary(
name = "my_service",
srcs = ["my_service.rs"],
deps = [":rust"] + GRPC_COMPILE_DEPS,
deps = [":rust"],
)
```
""",
Expand Down
13 changes: 10 additions & 3 deletions proto/repositories.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,14 @@ load("@bazel_tools//tools/build_defs/repo:utils.bzl", "maybe")
load("//proto/raze:crates.bzl", "rules_rust_proto_fetch_remote_crates")

# buildifier: disable=unnamed-macro
def rust_proto_repositories():
"""Declare dependencies needed for proto compilation."""
def rust_proto_repositories(register_default_toolchain = True):
"""Declare dependencies needed for proto compilation.

Args:
register_default_toolchain (bool, optional): If True, the default [rust_proto_toolchain](#rust_proto_toolchain)
(`@rules_rust//proto:default-proto-toolchain`) is registered. This toolchain requires a set of dependencies
that were generated using [cargo raze](https://github.com/google/cargo-raze). These will also be loaded.
"""
maybe(
http_archive,
name = "com_google_protobuf",
Expand Down Expand Up @@ -69,4 +75,5 @@ def rust_proto_repositories():
rules_rust_proto_fetch_remote_crates()

# Register toolchains
native.register_toolchains(str(Label("//proto:default-proto-toolchain")))
if register_default_toolchain:
native.register_toolchains(str(Label("//proto:default-proto-toolchain")))
38 changes: 30 additions & 8 deletions proto/toolchain.bzl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,21 @@
# buildifier: disable=bzl-visibility
load("//rust/private:utils.bzl", "name_to_crate_name")

def generated_file_stem(f):
basename = f.rsplit("/", 2)[-1]
def generated_file_stem(file_path):
"""Returns the basename of a file without any extensions.

Example:
```python
content.append("pub mod %s;" % _generated_file_stem(f))
```

Args:
file_path (string): A path to a file

Returns:
string: The file stem of the filename
"""
basename = file_path.rsplit("/", 2)[-1]
basename = name_to_crate_name(basename)
return basename.rsplit(".", 2)[0]

Expand Down Expand Up @@ -106,10 +119,12 @@ def rust_generate_proto(

def _rust_proto_toolchain_impl(ctx):
return platform_common.ToolchainInfo(
protoc = ctx.executable.protoc,
proto_plugin = ctx.file.proto_plugin,
grpc_plugin = ctx.file.grpc_plugin,
edition = ctx.attr.edition,
grpc_compile_deps = ctx.attr.grpc_compile_deps,
grpc_plugin = ctx.file.grpc_plugin,
proto_compile_deps = ctx.attr.proto_compile_deps,
proto_plugin = ctx.file.proto_plugin,
protoc = ctx.executable.protoc,
)

# Default dependencies needed to compile protobuf stubs.
Expand All @@ -124,22 +139,29 @@ GRPC_COMPILE_DEPS = PROTO_COMPILE_DEPS + [
Label("//proto/raze:tls_api_stub"),
]

# TODO(damienmg): Once bazelbuild/bazel#6889 is fixed, reintroduce
# proto_compile_deps and grpc_compile_deps and remove them from the
# rust_proto_library and grpc_proto_library.
rust_proto_toolchain = rule(
implementation = _rust_proto_toolchain_impl,
attrs = {
"edition": attr.string(
doc = "The edition used by the generated rust source.",
default = "2015",
),
"grpc_compile_deps": attr.label_list(
doc = "The crates the generated grpc libraries depends on.",
cfg = "target",
default = GRPC_COMPILE_DEPS,
),
"grpc_plugin": attr.label(
doc = "The location of the Rust protobuf compiler plugin to generate rust gRPC stubs.",
allow_single_file = True,
cfg = "exec",
default = Label("//proto:protoc_gen_rust_grpc"),
),
"proto_compile_deps": attr.label_list(
doc = "The crates the generated protobuf libraries depends on.",
cfg = "target",
default = PROTO_COMPILE_DEPS,
),
"proto_plugin": attr.label(
doc = "The location of the Rust protobuf compiler plugin used to generate rust sources.",
allow_single_file = True,
Expand Down
Loading