RFC: Sync Envoy protobufs to a nested proto module. #723
Conversation
jpeach
force-pushed
the
sync-to-protos-module
branch
from
aea7512
to
459ee5a
Compare
There was a problem hiding this comment.
+1 to this though I think it also needs a change (can maybe be a follow up to this PR) to switch the root module and code to import and reference this nested module, especially if we stop syncing the protos to the original package
if we want to be more conservative we could sync to both locations until the next release, and then stop syncing to the original after that; not sure how that will work with the actual generation bits since I haven't tried it (maybe change the go_package to the package path containing protos but also do a run where we do an ad-hoc sed replace on that option?), so maybe more headache than we want to deal with
Yup, I was assuming that we would make that change in #714. |
|
I tested the Envoy build with the import path change, and AFAICT Bazel doesn't do anything with the For example: Doesn't change the output path: Doesn't change the import path: |
|
OK, this diff seems to do what we want: |
Create a new "proto" directory for publishing the Envoy API protobufs. This lets core and contrib protobuf APIs be packaged in the same module, which is versioned independently of the main Go control plane. Signed-off-by: James Peach <jpeach@apache.org>
jpeach
force-pushed
the
sync-to-protos-module
branch
from
459ee5a
to
a2de5b5
Compare
|
I updated this to assume the changes in envoyproxy/envoy#28041 (i.e. the Envoy scripts generate Go code to the "proto" subdirectory). |
|
What happens when we import the new version of go-control-plane, but also import something like prometheus, grpc, etc that depend on an old version. Will we get duplicate protobufs which will cause panic? |
Yeh, the way I read it, if you imported the old and the new packages, the init functions in both would run and try to register the types. This would error with "foo is already registered", and the caller would panic (actually it would panic in If I'm reading it right, prometheus gets go-control-plane via grpc, so I bet that basically every consumer of go-control-plane would be the same. That said, the local experiments @sunjayBhatia tried with Contour didn't panic, so maybe it's OK. Definitely need to do some more experiments and understand what behavior to expect though. It probably depends on having a specific code path in the app that imports the same message via grpx and go-control-plane. |
But I'm not sure whether there is a robust way around this? Even if we move to new major versions of all the modules, the protobuf registry is still global and we can't update dependencies in lock-step. |
|
@howardjohn I verified that the protobuf registry does panic if you mix grpc with envoy APIs from a different package" package main
import (
"fmt"
_ "github.com/envoyproxy/go-control-plane/proto/envoy/config/core/v3"
_ "google.golang.org/grpc/xds"
"google.golang.org/protobuf/reflect/protoreflect"
"google.golang.org/protobuf/reflect/protoregistry"
)
func main() {
protoregistry.GlobalTypes.RangeMessages(
func(m protoreflect.MessageType) bool {
fmt.Println(m.Descriptor().FullName())
return true
},
)
}This results in: |
|
Closing this PR since it's clear that this approach won't work. |
Create a new "proto" directory for publishing the Envoy API protobufs. This lets core and contrib protobuf APIs be packaged in the same module, which is versioned independently of the main Go control plane.
This PR is an RFC to show people what the combined envoy+contrib module would look like. The GitHub envoy sync action ran on my fork, and you can see the results here.
Per the discussion in #714, this approach retains compatibility for existing users of go-control-plane, since we do no remove any existing code (we just stop updating it). Note that if everyone like this approach, we will need to update all the
option go_packagespecifications in the Envoy protobufs. This means that when projects do adopt the new module, they will need to edit all their import paths.