Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HttpMessageConverter based on Protostuff library [SPR-13203] #17795

Closed
spring-projects-issues opened this issue Jul 6, 2015 · 10 comments
Closed
Assignees
Labels
in: web status: declined type: enhancement

Comments

@spring-projects-issues
Copy link
Collaborator

spring-projects-issues commented Jul 6, 2015

Manish opened SPR-13203 and commented

We have a large application with an API layer on top. Current customers invoke the API using JSON or XML data packets. A few potential customers have asked us to add support for Google Protocol Buffers and BSON as well.

Since our @RestController s already exchange JSON and XML data using regular POJOs, we want to explore whether we can add Profobuf and BSON using the same POJOs. When we looked at existing Protobuf support in Spring WebMVC, we could see the ProtobufHttpMessageConverter class. However, this class only works with POJOs generated using protoc and hence does not work for our existing set up.

After searching around, we found the Protostuff library that can generate Protobuf messages with POJOs. We therefore created an HttpMessageConverter using Protostuff. As a bonus, we got BSON support through a format called smile that is also supported by Protostuff.

I would imagine that it would be a common need for other projects to integrate Protobuf support in an existing API layer. Would there be interest in adding Protostuff support to Spring WebMVC? If yes, I can contribute code to the project along with its necessary tests. I have attached our current message converter with this ticket for review and comments. If this is acceptable, I will submit a pull request on Github.


Affects: 4.1.7

Attachments:

Issue Links:

7 votes, 11 watchers

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Jul 6, 2015

Brian Clozel commented

Hi Manish

This is indeed an interesting feature.
One question - how do you achieve communication with 3rd party clients (Java, python, etc) if you're not providing them with *.proto files?
Are all those clients using the same Java POJOs?

As the current support for protobuf covers quite a lot already, I'm not scheduling this issue for the next version, but I'm leaving it open to see if people are interested in it. We definitely consider issues with strong use cases and votes from the community.

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Jul 6, 2015

Manish commented

Hi Brian

Thanks for the quick review. We have a DTO layer that comprises exclusively of request and response POJO pairs, one for each API endpoint. We currently package this as a JAR, use it within our own WAR and distribute it to customers using Java to connect to our API. We are looking to generate equivalent .NET classes using a code generator but that will come later as none of our clients (connecting through the API) is currently on .NET.

The idea is that if POJO serialization works with Protostuff, we will just ask clients to use Protostuff instead of the native Protobuf library with our DTO JAR. For future customers on .NET, there is also the option of hosting the JAR directly in IKVM and having the .NET code use the classes in the JAR through IKVM. I have used this option in the past for large projects so I know it works well. Therefore, there shouldn't be a pressing need for the .proto files for teams using Java or .NET.

Python and other languages may be a bit of a challenge but we work predominantly with customers who have their own systems written in Java (or .NET in some rare cases) so this may not be a big challenge for us. The Protostuff home page indicates (documentation on its Maven plugin) that .proto files can be generated as part of the build process, if required. I will try this out to make sure that the files can be generated for the common cases at least.

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Jul 15, 2015

hakamairi commented

Brian Clozel There are more reasons to make a full protostuff http converter.

  1. protobuf-java-format has not been released since May 22, 2011, last commit on Oct 20, 2011.
  2. It has a lot of open issues like this, preventing me from using this library for supporting XML.
  3. protostuff at the other hand is maintained.
  4. Provides a lot more formats.
  5. Doesn't have XML format issues.

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Jul 15, 2015

Brian Clozel commented

Interesting.
With this library not being actively maintained anymore and the soon to be released protobuf 3.0, we might want to address this issue.

What's your take on this Alex Antonov?

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Jul 15, 2015

Alex Antonov commented

@bclozel, this seems like a good idea to support both, but even though I agree that the ProtobufFormat library has not had any commits lately, it still is very useful, as, unlike Protostuff, it simply provides an alternative marshaling/serialization of the existing com.google.protobuf.Message objects to JSON or XML. There is also an alternative, but also very stale, https://github.com/sijuv/protobuf-codec that does similar stuff.

The difference between those 2 and Protostuff is that Protostuff actually replaces the Google's protoc compiler, and instead does their own code/schema generation, sometimes based from *.proto files, sometimes that is done at runtime using Java class metadata. This, in my mind, is a significant shift for people who have been using plain-old Google Protobuf, as they would not be required to switch to use Protostuff compilers, code generators and api.

I would welcome the addition of ProtostuffHttpMessageConverter, but not as a replacement, but as an addition to ProtobufHttpMessageConverter.

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Sep 22, 2015

Brian Clozel commented

Note: protobuf-java-format/ is now read-only on google code, but a new JsonFormat class has been added to the main protobuf project (see here).

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Nov 28, 2015

Kostiantyn Shchepanovskyi commented

If you need assistance with protostuff integration, please let me know.

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Nov 28, 2015

hakamairi commented

Brian Clozel Does it include the same changes as Protobuf-Java-Format available in mvn repository? Two new versions are available 1.3 and 1.4, and starting from 1.3 they have breaking changes in regards to Spring Message Converter using it.

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Nov 28, 2015

Juergen Hoeller commented

Guys, from my perspective, let's determine what we can do about Protostuff in the 4.3 timeframe, along with Protobuf v3 (#18166). Note that this is not a commitment for definitive inclusion yet; in particular, we need clear signals that Protostuff remains actively developed.

Juergen

@spring-projects-issues
Copy link
Collaborator Author

spring-projects-issues commented Nov 28, 2015

Brian Clozel commented

hakamairi, right now I think this issue deals with a different MessageConverter.
If you're interested in Google's protobuf support, please take a look and comment/vote on #18166.

@spring-projects-issues spring-projects-issues added status: declined type: enhancement in: web labels Jan 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
in: web status: declined type: enhancement
Projects
None yet
Development

No branches or pull requests

2 participants