Protobufs for brymck.io. This is both something I use both for personal projects and to demonstrate professionally how you can organize service definitions in a robust way.
This library is a monorepo housing Protobuf definitions. Its goal is to allow myself and contributors to define services that are
- Easy to write and maintain
- Efficient in production
- Easy for others to discover
- Easy for downstream users, both service developers and API consumers, to use
To accomplish these goals, I use Google's Protobufs as the IDL (interface description language) and gRPC as our RPC framework. Protobufs are language-neutral and come with a rich ecosystem of tools for code generation that allow serializing structured data. gRPC is a high performance RPC (remote procedure call) framework that supports multiple languages, using Protobufs for service definitions. To ensure we follow common practices, we lint and compile Protobufs with Prototool.
When the full CI/CD process is run, it will:
- Lint Protobuf definitions
- Generate, build and deploy Go, Java, Node.js and Python libraries for each Protobuf package
- Generate, build and deploy a Protobuf registry web site
The repo is laid out as follows:
.github/
- GitHub Actions CI/CD configurationbin/
- Python helper scripts for the build processconfig/
- Configuration for Google Cloud Endpoints, Maven, etc.proto/
- Protobuf service definition filestemplates/
- Templates for code generation not handled byprotoc
Makefile
- Directives for building and deploying libraries
Prerequisites:
To generate all code and build the resulting libraries (in some cases installing them locally), run:
make
This will build the following targets:
make dependencies
- Create Protobuf descriptor sets and dependency lists for each packagemake generate
- Use Prototool (and Proto Registry) to generate codemake package
- Reorganize code into directories for each language + package that contain complete package descriptions, using Jinja for templatingmake build
- Build each package
Note that package
and build
can all be appended with -go
, -java
, -node
and -python
to only run the
language-specific sections of each.
Create a new Protobuf file with basic information filled in based on the contents of proto/prototool.yaml
. Follow the
convention of proto/brymck/<package>/v<version>/<service>.proto
.
Validate your Protobuf files.
In general this should only run as part of CI/CD. Similar to package
and build
, you can provide the
language-appropriate suffix to this target to only deploy for that language.
Remove any generated files.