Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

shade json4s in scalalistener #6

Merged
merged 2 commits into from
Jan 14, 2022
Merged

shade json4s in scalalistener #6

merged 2 commits into from
Jan 14, 2022

Conversation

rccern
Copy link
Contributor

@rccern rccern commented Dec 7, 2021

Spark 3.2 changed version of json4s, shading helps have a single codebase compatible for spark 2.4, 3.0, 3.1, 3.2

LucaCanali
LucaCanali previously approved these changes Dec 8, 2021
Copy link

@LucaCanali LucaCanali left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand this has been tested on a swan test instance by @rccern and it's working OK.
Minor point: I would bump up the spark 3 version dependency, for example to Spark 3.2.0.
The change looks good to me.

@dungdm93
Copy link

@rccern are you missing add sbt plugin into project/plugins.sbt file

addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "1.1.0")

ref

scalalistener/build.sbt Outdated Show resolved Hide resolved
@rccern
Copy link
Contributor Author

rccern commented Dec 17, 2021

@dungdm93 thanks, I had added the plugins file in gitignore, but it didn't work because the directory above was excluded and put back org.json4s in the shaded class names, it makes it more easy to understand

@LucaCanali bumped spark to 3.2.0

@rccern rccern merged commit 2b9e647 into master Jan 14, 2022
@krishnan-r krishnan-r mentioned this pull request Jun 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants