Allow construction of ProtobufSchema using Jackson introspection functionality

[cowtowncoder]

Similar to existing csv and avro modules' functionality that allows construction of schemas given a single POJO definition, it would make to allow construction of ProtobufSchema from a POJO (class) definition.

[dvdkruk]

Is there currently any progress about this feature? If you not started on this yet, I would like to make it. However I'm new to Jackson and don't how hard it is to program this kind functionality. Maybe you can help me out?

[cowtowncoder]

I have not started actual work on it, although I did investigate possible ways, models and so forth.
But no code, so if you wanted to try working on it, that would be great.

I think that the main challenge should not be so much Jackson integration as just modeling; if you can create a useful model (intuitive to use, modify), and one that can be built both manually and from protoc definition, that would be mostly there. I can help with integration part; it could use existing abstractions (modified as needed) as target. That is, intermediate model would be used to construct pieces that are directly used by encoder/decoder. As is, we are basically using protobuf-parser's intermediate definition..

[dvdkruk]

Ok, I will go look more into this next week.

For now I already have made a parser with Xtext (called: Protolipse), which can serialize and generate .proto files to and from an Ecore-based model. However this solution is most likely too heavy for this extension. What is your opinion about this?

Also found protostuff-runtime-proto on github which actually does what we want, however it is made for Protostuff and so uses the internal schema model of Protostuff as intermediate model. This can maybe be a good base/starting point.

[cowtowncoder]

I did earlier use protostuff, but found protoparser package (https://github.com/square/protoparser) to be better option, which is why I use that for protoc parsing.
So I don't think protostuff package is usable as is, although some aspects could serve as inspiration possibly.

I don't deeply care about what is used, but ideally it should be relatively small package with no additional external dependencies.

The main problem with existing libraries is, in my opinion, that their intermediate models are not very good; either they are not modular, or they only allow creation from protoc, or in some cases they use/expose google's default protobuf package, which is somewhat counter-productive since we do not need most of the things (codec here is specifically a replacement).

It seems to me that existing packages should be fine for parsing of protoc; although if a simple grammar generator can produce a parser to use that's fine as well. But the main work is the next step, intermediate model. Parse trees are not adequate themselves, both because they do not fully resolve inter-dependencies (references are by ids, those need to be resolved to object references, just as an example), and because they are too tied to protoc representation, which is unwieldy for programmatic definition and access.

I am not familiar with xtext, but the main questions there would be integrating parser generation with build process (assuming it uses a source specification of some kind), and resulting parser size.

[dvdkruk]

This one can be closed now, right?

[cowtowncoder]

Right, yes.