New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
java.lang.UnsupportedOperationException caused by org.elasticsearch.hadoop.mr.EsInputFormat ? #338
Comments
I tried elasticsearch-hadoop-2.1.0.Beta3.jar and with the latest snapshot. |
Can you provide some background on what you are trying to do? What version of Spark and Python are you using? I assume you are trying to interact with ES through the Map/Reduce layer instead of the Java/Scala API correct? Thanks! |
OK, am newbie, not quite sure what I am doing, so please bear with me. Someone on spark mailing list suggested it could be a bug in EsInputFormat Versions: am trying to learn how to use PySpark to process data stored in Elasticseach. This is my code based on some examples I could find around: I start iPython as follows:
I run this with no errors:
I run this with no errors also:
which prints my document title. Then I try this which gives the exceptions I had pasted above:
|
I think I've found the problem and pushed a nightly build with it. Can you please download it and try it out ? it's on the 2.1.0.BUILD-SNAPSHOT which you can read about in the docs, here Thanks! |
Great! Now I get a much more helpful exception.. thanks a lot.
|
Thanks for the feedback. I assume the issue is fixed on the es-hadoop side. If that's not the case, please reopen the issue. By the way, I recommend using the native Scala/Java integration instead of the M/R layer as the former is much more performant. |
In Hadoop like environments (like Spark) the objects passed to consumers might be completely replaced and lead to strange exceptions. For this reason always create a new object under both MR APIs (old and new) Fix #338
Trying to figure out why this is always crashing.. someone pointed out might be a bug in
EsInputFormat ?
The text was updated successfully, but these errors were encountered: