Is the "gogoproto" dependency really necesary?

[valexandersaulys]

Hello all,

I'm trying to build out an adapter for use with a Spark / Hadoop setup to store prometheus data more long term while keeping it ready to be analyzable.

However, I'm running into an issue whereby the protocol buffer templates are written with a Go dependency called "gogoproto", which doesn't have any non-Go analogue. Hadoop, Spark, and the whole distributed big data stack (specifically I'm using the Cloudera flavor) uses the JVM and I'm limited in using go.

So my big question is, do we absolutely have to use gogoproto as a dependency? I'm willing to do the work here to eliminate it, but I'm not that familiar with go nor am I sure just how necesary it is.

Any help here would be nice, thanks!

[valexandersaulys]

This might not actually be necesary with OpenTSDB support as an example.

My thinking was to use something like Apache Kudu, but I can likely feed into OpenTSDB and then parse into Hadoop from there.

Anyways, I'm closing the issue now. Anybody who wants to revisit this is free to open it back up.

[brian-brazil]

It's just protobuf, which library we've chosen to use on the Go side shouldn't matter to you. As far as we're aware, our remote read/write protocol can be communicated to from Java.

[valexandersaulys]

That is definitely not the case.

When I try to generate python files for the protocol buffer API, I get library imports like this:

# example taken from types.proto, which becomes types_pb2.py
from gogoproto import gogo_pb2 as gogoproto_dot_gogo__pb2

This library does not exist in python, it only exists in Go. It gets generated because of how these proto files are written. Somebody in the CockroachDB github talked about a similar issue, which is how I found my answer personally. Granted, this was a while back, but I can't find anything about a "gogoproto" library in python. I'd be happy if you knew of one!

[fabxc]

With the state of things, not using gogoproto is simply not an option from a performance and code sanity perspective.
That said, it does not block interoperability from an API perspective – that's what protobuf is for ultimately. If you generate clients or servers with the schema, things will work. The annotations for language-specific libraries getting in the way seems common but needed to get sane code out of it.

The only reasonable answer right now is to copy the file, strip all annotations and potentially replace them with annotations for the language of your choice. It's an awful user experience. But that's for the protobuf/grpc community to figure out.

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.