-
Notifications
You must be signed in to change notification settings - Fork 29
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Incompatibilities with Spark because of json4s #363
Comments
@framis: I've changed the dependency for json4s, could you tell me if this fixes your issue? |
Sure, I'll check tomorrow. Thanks! |
@framis were you able to test it? |
I used com.algolia:scala-client_2.11:1.0.1-SNAPSHOT I managed to test it (both spark-submit and spark-shell, with Spark version 2.1). Now it compiles, which is better than last time, but I got this exception:
when trying to do
You can find the code I used here: https://github.com/framis/spark-algolia |
I made a PR on your code, it works for me locally: framis/spark-algolia#1 |
Great! We're good to go, then! |
Ok I'll merge this and release the new version |
Algolia Scala API client is not compatible with Spark (any version) because of an incompatible version of json4s (Spark#2.0.2 depends on json4s version 3.2.11, Algolia latest client 3.4 and those binaries are incompatible as expressed here). You certainly want to provide an artifact without the json4s dependency or using an older version for Spark compatibility.
cc @framis
The text was updated successfully, but these errors were encountered: