Permalink
Browse files

internal/language/proto: package mode

The proto extension now supports "package" mode. This can be enabled
with -proto=package on the command line or "# gazelle:proto package"
in a build file. In package mode, multiple proto_libraries will be
generated in a directory. Source files will be grouped by package.

To group source files by an option instead (i.e., option go_package),
the proto extension recognizes the proto_group command line flag and
directive.

The go extension generates go_proto_library rules based on the
proto_library rules generated by the proto extension (independent of
proto mode when possible).

Related bazelbuild/rules_go#1548
  • Loading branch information...
jayconrod committed Jul 2, 2018
1 parent 16c8357 commit 91e6a1e1fcb728056bb10a5cc0fc062f254e154e
Showing with 838 additions and 319 deletions.
  1. +95 −74 README.rst
  2. +1 −0 internal/go_repository.bzl
  3. +5 −0 internal/language/go/constants.go
  4. +9 −6 internal/language/go/fileinfo_go_test.go
  5. +177 −136 internal/language/go/generate.go
  6. +46 −0 internal/language/go/generate_test.go
  7. +53 −3 internal/language/go/package.go
  8. +1 −0 internal/language/go/testdata/proto_package_mode/BUILD.old
  9. +43 −0 internal/language/go/testdata/proto_package_mode/BUILD.want
  10. +5 −0 internal/language/go/testdata/proto_package_mode/bar1.proto
  11. +3 −0 internal/language/go/testdata/proto_package_mode/bar2.proto
  12. +5 −0 internal/language/go/testdata/proto_package_mode/foo1.proto
  13. +5 −0 internal/language/go/testdata/proto_package_mode/foo2.proto
  14. +3 −0 internal/language/go/testdata/proto_package_mode_extras/BUILD.old
  15. +54 −0 internal/language/go/testdata/proto_package_mode_extras/BUILD.want
  16. +5 −0 internal/language/go/testdata/proto_package_mode_extras/bar1.proto
  17. +3 −0 internal/language/go/testdata/proto_package_mode_extras/bar2.proto
  18. +5 −0 internal/language/go/testdata/proto_package_mode_extras/foo1.proto
  19. +5 −0 internal/language/go/testdata/proto_package_mode_extras/foo2.proto
  20. +1 −0 internal/language/go/testdata/proto_package_mode_extras/foo_test.go
  21. +17 −2 internal/language/proto/config.go
  22. +5 −4 internal/language/proto/constants.go
  23. +145 −62 internal/language/proto/generate.go
  24. +50 −14 internal/language/proto/generate_test.go
  25. +20 −18 internal/language/proto/package.go
  26. +5 −0 internal/language/proto/testdata/multiple_packages/default_mode/BUILD.want
  27. +3 −0 internal/language/proto/testdata/multiple_packages/default_mode/bar.proto
  28. +3 −0 internal/language/proto/testdata/multiple_packages/default_mode/foo.proto
  29. +6 −0 internal/language/proto/testdata/multiple_packages/package_mode/BUILD.old
  30. +17 −0 internal/language/proto/testdata/multiple_packages/package_mode/BUILD.want
  31. +3 −0 internal/language/proto/testdata/multiple_packages/package_mode/bar1.proto
  32. +3 −0 internal/language/proto/testdata/multiple_packages/package_mode/bar2.proto
  33. +3 −0 internal/language/proto/testdata/multiple_packages/package_mode/foo1.proto
  34. +3 −0 internal/language/proto/testdata/multiple_packages/package_mode/foo2.proto
  35. +2 −0 internal/language/proto/testdata/multiple_packages/package_mode_group/BUILD.old
  36. +17 −0 internal/language/proto/testdata/multiple_packages/package_mode_group/BUILD.want
  37. +3 −0 internal/language/proto/testdata/multiple_packages/package_mode_group/bar1.proto
  38. +3 −0 internal/language/proto/testdata/multiple_packages/package_mode_group/bar2.proto
  39. +3 −0 internal/language/proto/testdata/multiple_packages/package_mode_group/foo1.proto
  40. +3 −0 internal/language/proto/testdata/multiple_packages/package_mode_group/foo2.proto
View
@@ -230,77 +230,82 @@ Subdirectories will be processed recursively.
The following flags are accepted:
+------------------------------------------+-----------------------------------+
| **Name** | **Default value** |
+==========================================+===================================+
| :flag:`-build_file_name file1,file2,...` | :value:`BUILD.bazel,BUILD` |
+------------------------------------------+-----------------------------------+
| Comma-separated list of file names. Gazelle recognizes these files as Bazel |
| build files. New files will use the first name in this list. Use this if |
| your project contains non-Bazel files named ``BUILD`` (or ``build`` on |
| case-insensitive file systems). |
+------------------------------------------+-----------------------------------+
| :flag:`-build_tags tag1,tag2` | |
+------------------------------------------+-----------------------------------+
| List of Go build tags Gazelle will consider to be true. Gazelle applies |
| constraints when generating Go rules. It assumes certain tags are true on |
| certain platforms (for example, ``amd64,linux``). It assumes all Go release |
| tags are true (for example, ``go1.8``). It considers other tags to be false |
| (for example, ``ignore``). This flag overrides that behavior. |
| |
| Bazel may still filter sources with these tags. Use |
| ``bazel build --features gotags=foo,bar`` to set tags at build time. |
+------------------------------------------+-----------------------------------+
| :flag:`-external external|vendored` | :value:`external` |
+------------------------------------------+-----------------------------------+
| Determines how Gazelle resolves import paths. May be :value:`external` or |
| :value:`vendored`. Gazelle translates Go import paths to Bazel labels when |
| resolving library dependencies. Import paths that start with the |
| ``go_prefix`` are resolved to local labels, but other imports |
| are resolved based on this mode. In :value:`external` mode, paths are |
| resolved using an external dependency in the WORKSPACE file (Gazelle does |
| not create or maintain these dependencies yet). In :value:`vendored` mode, |
| paths are resolved to a library in the vendor directory. |
+------------------------------------------+-----------------------------------+
| :flag:`-go_prefix example.com/repo` | |
+------------------------------------------+-----------------------------------+
| A prefix of import paths for libraries in the repository that corresponds to |
| the repository root. Gazelle infers this from the ``go_prefix`` rule in the |
| root BUILD.bazel file, if it exists. If not, this option is mandatory. |
| |
| This prefix is used to determine whether an import path refers to a library |
| in the current repository or an external dependency. |
+------------------------------------------+-----------------------------------+
| :flag:`-known_import example.com` | |
+------------------------------------------+-----------------------------------+
| Skips import path resolution for a known domain. May be repeated. |
| |
| When Gazelle resolves an import path to an external dependency, it attempts |
| to discover the remote repository root over HTTP. Gazelle skips this |
| discovery step for a few well-known domains with predictable structure, like |
| golang.org and github.com. This flag specifies additional domains to skip, |
| which is useful in situations where the lookup would fail for some reason. |
+------------------------------------------+-----------------------------------+
| :flag:`-mode fix|print|diff` | :value:`fix` |
+------------------------------------------+-----------------------------------+
| Method for emitting merged build files. |
| |
| In ``fix`` mode, Gazelle writes generated and merged files to disk. In |
| ``print`` mode, it prints them to stdout. In ``diff`` mode, it prints a |
| unified diff. |
+------------------------------------------+-----------------------------------+
| :flag:`-proto default|legacy|disable` | :value:`default` |
+------------------------------------------+-----------------------------------+
| Determines how Gazelle should generate rules for .proto files. See details |
| in `Directives`_ below. |
+------------------------------------------+-----------------------------------+
| :flag:`-repo_root dir` | |
+------------------------------------------+-----------------------------------+
| The root directory of the repository. Gazelle normally infers this to be the |
| directory containing the WORKSPACE file. |
| |
| Gazelle will not process packages outside this directory. |
+------------------------------------------+-----------------------------------+
+-----------------------------------------------+-----------------------------------+
| **Name** | **Default value** |
+===============================================+===================================+
| :flag:`-build_file_name file1,file2,...` | :value:`BUILD.bazel,BUILD` |
+-----------------------------------------------+-----------------------------------+
| Comma-separated list of file names. Gazelle recognizes these files as Bazel |
| build files. New files will use the first name in this list. Use this if |
| your project contains non-Bazel files named ``BUILD`` (or ``build`` on |
| case-insensitive file systems). |
+-----------------------------------------------+-----------------------------------+
| :flag:`-build_tags tag1,tag2` | |
+-----------------------------------------------+-----------------------------------+
| List of Go build tags Gazelle will consider to be true. Gazelle applies |
| constraints when generating Go rules. It assumes certain tags are true on |
| certain platforms (for example, ``amd64,linux``). It assumes all Go release |
| tags are true (for example, ``go1.8``). It considers other tags to be false |
| (for example, ``ignore``). This flag overrides that behavior. |
| |
| Bazel may still filter sources with these tags. Use |
| ``bazel build --features gotags=foo,bar`` to set tags at build time. |
+-----------------------------------------------+-----------------------------------+
| :flag:`-external external|vendored` | :value:`external` |
+-----------------------------------------------+-----------------------------------+
| Determines how Gazelle resolves import paths. May be :value:`external` or |
| :value:`vendored`. Gazelle translates Go import paths to Bazel labels when |
| resolving library dependencies. Import paths that start with the |
| ``go_prefix`` are resolved to local labels, but other imports |
| are resolved based on this mode. In :value:`external` mode, paths are |
| resolved using an external dependency in the WORKSPACE file (Gazelle does |
| not create or maintain these dependencies yet). In :value:`vendored` mode, |
| paths are resolved to a library in the vendor directory. |
+-----------------------------------------------+-----------------------------------+
| :flag:`-go_prefix example.com/repo` | |
+-----------------------------------------------+-----------------------------------+
| A prefix of import paths for libraries in the repository that corresponds to |
| the repository root. Gazelle infers this from the ``go_prefix`` rule in the |
| root BUILD.bazel file, if it exists. If not, this option is mandatory. |
| |
| This prefix is used to determine whether an import path refers to a library |
| in the current repository or an external dependency. |
+-----------------------------------------------+-----------------------------------+
| :flag:`-known_import example.com` | |
+-----------------------------------------------+-----------------------------------+
| Skips import path resolution for a known domain. May be repeated. |
| |
| When Gazelle resolves an import path to an external dependency, it attempts |
| to discover the remote repository root over HTTP. Gazelle skips this |
| discovery step for a few well-known domains with predictable structure, like |
| golang.org and github.com. This flag specifies additional domains to skip, |
| which is useful in situations where the lookup would fail for some reason. |
+-----------------------------------------------+-----------------------------------+
| :flag:`-mode fix|print|diff` | :value:`fix` |
+-----------------------------------------------+-----------------------------------+
| Method for emitting merged build files. |
| |
| In ``fix`` mode, Gazelle writes generated and merged files to disk. In |
| ``print`` mode, it prints them to stdout. In ``diff`` mode, it prints a |
| unified diff. |
+-----------------------------------------------+-----------------------------------+
| :flag:`-proto default|package|legacy|disable` | :value:`default` |
+-----------------------------------------------+-----------------------------------+
| Determines how Gazelle should generate rules for .proto files. See details |
| in `Directives`_ below. |
+-----------------------------------------------+-----------------------------------+
| :flag:`-proto_group group` | :value:`""` |
+-----------------------------------------------+-----------------------------------+
| Determines the proto option Gazelle uses to group .proto files into rules |
| when in ``package`` mode. See details in `Directives`_ below. |
+-----------------------------------------------+-----------------------------------+
| :flag:`-repo_root dir` | |
+-----------------------------------------------+-----------------------------------+
| The root directory of the repository. Gazelle normally infers this to be the |
| directory containing the WORKSPACE file. |
| |
| Gazelle will not process packages outside this directory. |
+-----------------------------------------------+-----------------------------------+
``update-repos``
~~~~~~~~~~~~~~~~
@@ -475,9 +480,12 @@ The following directives are recognized:
+------------------------------------------+-----------------------------------+
| Tells Gazelle how to generate rules for .proto files. Valid values are: |
| |
| * ``default``: ``proto_library``, ``go_proto_library``, ``go_grpc_library``, |
| and ``go_library`` rules are generated using |
| ``@io_bazel_rules_go//proto:def.bzl``. This is the default mode. |
| * ``default``: ``proto_library``, ``go_proto_library``, and ``go_library`` |
| rules are generated using ``@io_bazel_rules_go//proto:def.bzl``. Only one |
| of each rule may be generated per directory. This is the default mode. |
| * ``package``: multiple ``proto_library`` and ``go_proto_library`` rules |
| may be generated in the same directory. .proto files are grouped into |
| rules based on their package name or another option (see ``proto_group``). |
| * ``legacy``: ``filegroup`` rules are generated for use by |
| ``@io_bazel_rules_go//proto:go_proto_library.bzl``. ``go_proto_library`` |
| rules must be written by hand. Gazelle will run in this mode automatically |
@@ -494,6 +502,19 @@ The following directives are recognized:
| ``@io_bazel_rules_go//proto:go_proto_library.bzl`` is loaded, Gazelle |
| will run in ``legacy`` mode. |
+------------------------------------------+-----------------------------------+
| :direc:`proto_group` | :value:`""` |
+------------------------------------------+-----------------------------------+
| Specifies an option that Gazelle can use to group .proto files into rules |
| when in ``package`` mode. For example, when set to ``go_package``, .proto |
| files with the same ``option go_package`` will be grouped together. |
| |
| When this directive is set to the empty string, Gazelle will group packages |
| by their proto package statement. |
| |
| Rule names are generated based on the last run of identifier characters |
| in the package name. For example, if the package is ``"foo/bar/baz"``, the |
| ``proto_library`` rule will be named ``baz_proto``. |
+------------------------------------------+-----------------------------------+
Keep comments
~~~~~~~~~~~~~
@@ -164,6 +164,7 @@ go_repository = repository_rule(
values = [
"",
"default",
"package",
"disable",
"legacy",
],
@@ -19,4 +19,9 @@ const (
// legacyProtoFilegroupName is the anme of a filegroup created in legacy
// mode for libraries that contained .pb.go files and .proto files.
legacyProtoFilegroupName = "go_default_library_protos"
// wellKnownTypesGoPrefix is the import path for the Go repository containing
// pre-generated code for the Well Known Types.
wellKnownTypesGoPrefix = "github.com/golang/protobuf"
// wellKnownTypesPkg is the package name for the predefined WKTs in rules_go.
wellKnownTypesPkg = "proto/wkt"
)
@@ -365,13 +365,16 @@ import "C"
c.RepoRoot = repo
gc := getGoConfig(c)
gc.prefix = "example.com/repo"
got := buildPackage(c, sub, "sub", []string{"sub.go"}, nil, nil, false, "", nil)
pkgs, _ := buildPackages(c, sub, "sub", []string{"sub.go"}, false)
got, ok := pkgs["sub"]
if !ok {
t.Fatal("did not build package 'sub'")
}
want := &goPackage{
name: "sub",
dir: sub,
rel: "sub",
importPath: "example.com/repo/sub",
library: goTarget{cgo: true},
name: "sub",
dir: sub,
rel: "sub",
library: goTarget{cgo: true},
}
want.library.sources.addGenericString("sub.go")
want.library.copts.addGenericString("-Isub/..")
Oops, something went wrong.

0 comments on commit 91e6a1e

Please sign in to comment.