New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

protoc-gen-go: fully generate marshaling and unmarshaling code #280

Open
awalterschulze opened this Issue Jan 18, 2017 · 5 comments

Comments

Projects
None yet
5 participants
@awalterschulze

awalterschulze commented Jan 18, 2017

The Conversation started in #276

Is there a possibility of golang/protobuf starting to generate some marshaling and unmarshaling code?
I would recommend this over any use of unsafe, since then protocol buffers doesn't just play nice with appengine, but also gopherjs.
Also generated code is much faster, which is one of the reasons people flock to gogoprotobuf.
It would also be great if the generated code didn't depend on the proto package.
But this is just my personal wish, one less dependency for users of a package using protobufs.

  • Walter Schulze

We're looking into ways to speed up marshal and unmarshal. Entirely generated code seems unlikely to be the right end point (too much code, not enough performance win), but we're evaluating multiple points in the solution space. Code not depending on the proto package at all is an interesting idea, although that implies even more generated code. I don't think that was on our radar.

@awalterschulze

This comment has been minimized.

awalterschulze commented Jan 18, 2017

I really think the amount of speed gained is significant enough to warrant the extra code generation for marshal and unmarshal.

Here is an independant benchmark
https://github.com/alecthomas/go_serialization_benchmarks

BenchmarkGoprotobufMarshal-8                2000000        746 ns/op      312 B/op        4 allocs/op
BenchmarkGoprotobufUnmarshal-8              2000000        978 ns/op      432 B/op        9 allocs/op
BenchmarkGogoprotobufMarshal-8             10000000        211 ns/op       64 B/op        1 allocs/op
BenchmarkGogoprotobufUnmarshal-8            5000000        289 ns/op       96 B/op        3 allocs/op

Also once you have those methods, the extra generated code required to not use the proto package is very little.

@rsc rsc changed the title from Again: Why not generate marshaling and unmarshaling code to generate marshaling and unmarshaling code Jan 18, 2017

@rsc

This comment has been minimized.

Contributor

rsc commented Jan 18, 2017

@awalterschulze

This comment has been minimized.

awalterschulze commented Jan 18, 2017

This surely cannot be news?

But if it is, this is gogoprotobuf, it does code generation (and other gadgets) for protobuffers for go, since 2013.
https://github.com/gogo/protobuf

It has users, including Kubernetes
https://github.com/gogo/protobuf#users

@awalterschulze

This comment has been minimized.

awalterschulze commented Jan 18, 2017

One of the newer things you can do with gogoprotobuf.
Using this command you can still import github.com/golang/protobuf/proto, but get the generated marshaler, size and unmarshaler. So only your code generation steps depends on gogoprotobuf.

go install github.com/gogo/protobuf/protoc-gen-gofast
protoc --gofast_out=. my.proto

Maybe this can give you an idea of what generating code in golang/protobuf would be like.

@johanbrandhorst

This comment has been minimized.

Member

johanbrandhorst commented Feb 18, 2018

Bump, this discussion should be revisited, as it still has no satisfying conclusion. Recent discussion implies this still hasn't been closed as a possibility. What are the current thoughts on this?

@dsnet dsnet changed the title from generate marshaling and unmarshaling code to protoc-gen-go: fully generate marshaling and unmarshaling code Mar 8, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment