go get for Protobuf - the missing link in your pb toolchain.
pbtree downloads dependencies for your protofiles and organizes them into a
single directory tree for later building.
This tool should be used before protoc and any proto linters.
- pbtree
- Installation
- Quick start
- Pulling protofiles from other projects
- Canonical import format
- Full documentation
Protobuf files use C-style imports - that is, any relative import will work if
protoc is able to find it under any of include paths (passed via -I flag).
That makes the building process too brittle - you need to supply a lot of -I’s
and clone all the repos to proper paths.
pbtree attempts to solve that problem by rewriting all imports to a single
URI-like import style resembling Go’s import rules; the format looks like
git.corp/my/repo!/path/file.proto. Afterwards, it downloads dependencies and
organizes them into a single file tree.
This is a first public release, but it is already used in production.
pbtree covers most of the usecases of the companies I’ve been working with.
If it doesn’t work for you - create an issue and we’ll get there :)
TODO:
- recursive versioning of dependencies, #1
- global cache for dependencies pulled via HTTP
- …
Grab the latest release here, or fetch it via go tool:
GO111MODULE=on go get github.com/utrack/pbtree@latest
Navigate to your repository and create a new pbtree project, passing full repo name via --module:
» mkdir super-project && cd super-project
» pbtree init github.com/me/super-project
2020/07/16 17:45:07 new config is ready at '.pbtree.yml', edit away or see 'pbtree help add'
Add directory with protofiles:
» mkdir protos && curl -o protos/foo.proto "https://gist.githubusercontent.com/utrack/0cac21b0ca1fafb96ef82afe15418037/raw/5ae54db359036736deaf020de8f205154fa57eaa/foo.proto"
» pbtree add ./protos
Build your proto file tree:
» pbtree build
fetcher: using http fetcher for 'github.com/google/protobuf'
Check the generated tree:
» tree -a
.
├── .pbtree.yml
├── protos
│ └── foo.proto
└── vendor.pbtree
└── github.com
├── google
│ └── protobuf!
│ └── src
│ └── google
│ └── protobuf
│ └── timestamp.proto
└── me
└── super-project!
└── protos
└── foo.proto
Now, you can use protoc to generate your proto(s) with a single command:
» protoc -I./vendor.pbtree --go_out=. ./vendor.pbtree/github.com/me/super-project\!/protos/foo.proto
Sometimes pbtree won’t be able to figure out your imports; you can either
change them to Canonical import format by hand or Map your imports without
changing files themselves.
You can pull any 3rd-party protofiles to generate clients for services
described in other repos. To pull them, use pbtree get:
» pbtree get 'github.com/googleapis/googleapis!/google/type/datetime.proto'
fetcher: using http fetcher for 'github.com/googleapis/googleapis'
INFO file successfully added, don't forget to call 'pbtree build'!
» pbtree build
fetcher: using http fetcher for 'github.com/googleapis/googleapis'
fetcher: using http fetcher for 'github.com/google/protobuf'
» tree -al
.
├── .pbtree.yaml
├── protos
│ └── foo.proto
└── vendor.pbtree
└── github.com
├── google
│ └── protobuf!
│ └── src
│ └── google
│ └── protobuf
│ ├── duration.proto
│ └── timestamp.proto
├── googleapis
│ └── googleapis!
│ └── google
│ └── type
│ └── datetime.proto
└── me
└── super-project!
└── protos
└── foo.proto
Any import is converted to a single format that looks like
repository!/dir/file.proto, e.g.:
git.enterprise.com/my/project!/api/file.proto
github.com/google/protobuf!/src/google/protobuf/timestamp.proto
etc.
You can use this import format in your protofiles:
syntax = "proto3";
import "foo.com/bar/baz!/dir/file.proto"
For any other formats, pbtree will try to guesstimate what you want. See
ImportPathDiscovery for detailed info.
Check out the wiki for more.