You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
the student despair is real, i also had this issue, luckily for me i solved it on my own , the cause of the issue is that spark 3 is compiled in scala 2.12 , and the connectors in elasticsearch are compiled in 2.11, since elasticsearch website doesn't have that in their site (for some reason like wtf ?), u can search that in maven and install it manually, mvn dependency:get -DgroupId=org.elasticsearch -DartifactId=elasticsearch-spark-30_2.12 -Dversion=8.4.3 -Dtransitive=false
then check the local maven repo /home/YourUserName/.m2/repository
either that or get a spark that is compiled in scala 2.11, which is spark 2.x versions but then the python version won't work cuz u will need python 3.4 or something which no longer supported , it's such a mean problem
i am getting this error
Py4JJavaError: An error occurred while calling o226.save.
: java.lang.NoClassDefFoundError: scala/Product$class
in this step
write ratings data
ratings.write.format("es").save("ratings")
num_ratings_es = es.count(index="ratings")['count']
num_ratings_df = ratings.count()
check write went ok
print("Dataframe count: {}".format(num_ratings_df))
print("ES index count: {}".format(num_ratings_es))
plz help me
The text was updated successfully, but these errors were encountered: