Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add xtra flag to send json on Kafka transport #78

Open
gjelu opened this issue May 25, 2020 · 2 comments
Open

add xtra flag to send json on Kafka transport #78

gjelu opened this issue May 25, 2020 · 2 comments

Comments

@gjelu
Copy link

gjelu commented May 25, 2020

The protobuffer has support for marshaling protobuffer messages as json: google.golang.org/protobuf/encoding/protojson

For some popular tools, like logstash (that are more text oriented), having json would be a better option as message format. (See logstash-plugins/logstash-codec-protobuf#15). At least it will make it a lot easier to share data between (legacy) systems.

Basically the impact would be minimum: in the Kafka transport the protobuffer message is marshaled as json format instead of wire format based on a flag. Fields of type bytes are then converted as base64 strings which is, for some systems, safer.

I've implemented a working version for this: gjelu@e2a0c56

Some extra required changes:

  • I am using protojson from the protobuf APIv2, so I need to convert the v1 message to a v2 message.
  • for this I need to upgrade to github.com/golang/protobuf@v1.4.0

I can do a PR if it's feasible that this can be accepted? Or if you have any extra comments on this?

@Slepwin
Copy link

Slepwin commented Jan 15, 2021

It would be very useful feature for me.

@mugugnu
Copy link

mugugnu commented Jan 29, 2021

I'd love to see this feature!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants