Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: Separate protobuf and json parsers #228

Closed
wi101 opened this issue Dec 15, 2022 · 2 comments · Fixed by #245
Closed

Proposal: Separate protobuf and json parsers #228

wi101 opened this issue Dec 15, 2022 · 2 comments · Fixed by #245
Assignees

Comments

@wi101
Copy link

wi101 commented Dec 15, 2022

Hey 👋

We're using teleproto in a project to convert to protobuf and we use Spark. We got this error:

ERROR SparkHadoopWriter: Aborting job job_20221214135935241508088406623081_0074.
org.apache.spark.SparkException: Job aborted due to stage failure: Task serialization failed: java.lang.NoClassDefFoundError: org/json4s/JsonAST$JValue
java.lang.NoClassDefFoundError: org/json4s/JsonAST$JValue

Because Spark is using an older json4s version
we had to exclude the "com.thesamet.scalapb" %% "scalapb-json4s to make it work.

  • Currently:
    VersionedModelReader has fromJson and fromProto
    and VersionedModelWriter has toJson and toProto

  • Suggestion:
    To have a separate module for json and for proto that will be published in separate jars

@saeltz
Copy link
Member

saeltz commented Feb 21, 2023

Hey Wiem,

thanks for your proposal!

We're actively investigating support for Scala 3 at the moment and don't want to spend too many resources on the Scala 2 version.

An easier alternative might be to declare the dependency on scalapb-json4s as optional in this library.
Do you think this would solve your issue? If so, I'd happily do the change. If not, I'll take another look at modularising the library.

@saeltz saeltz self-assigned this Feb 21, 2023
@wi101
Copy link
Author

wi101 commented Feb 24, 2023

I think the suggested alternative with optional dependency would be helpful :) thank you!

saeltz added a commit that referenced this issue Mar 2, 2023
* #228: Declare dependency on json4s optional

* Add to README
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants