Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Saving case class with mapping id error #384

Closed
simonlei opened this issue Feb 28, 2015 · 3 comments
Closed

Saving case class with mapping id error #384

simonlei opened this issue Feb 28, 2015 · 3 comments
Labels
enhancement :Spark v2.1.0.Beta4

Comments

@simonlei
Copy link

Start the spark-shell with today's snapshot jar:
bin/spark-shell --jars ~/Download/elasticsearch-hadoop-2.1.0.BUILD-SNAPSHOT.jar

Then, run codes below:

    import org.elasticsearch.spark.rdd.EsSpark                        
    case class Trip(id:Integer, departure: String, arrival: String)               

    val upcomingTrip = Trip( 1, "OTP", "SFO")
    val lastWeekTrip = Trip( 2, "MUC", "OTP")

    val rdd = sc.makeRDD(Seq(upcomingTrip, lastWeekTrip))
    EsSpark.saveToEs(rdd, "spark/docs2", Map( "es.mapping.id"->"id"))

Then it will throw exceptions

This issue is related to [Spark] Case Class example failed after compile, but without the es.mapping.id, it works fine, so I create a new one.

Thanks.
costin added a commit that referenced this issue Mar 2, 2015
@costin
[SPARK] Add field extraction for case classes and JavaBeans
3e29910 
relates #384
@costin
Copy link
Member

costin commented Mar 2, 2015

@simonlei Pushed in master and the dev builds in master. can you please try them out?

Thanks,

costin added a commit that referenced this issue Mar 2, 2015
@costin
[SPARK] Add field extraction for case classes and JavaBeans
ed7f175 
relates #384
@simonlei
Copy link
Author

simonlei commented Mar 3, 2015

@costin Great! It works with the snapshot version elasticsearch-hadoop-2.1.0.BUILD-20150302.222111-319.jar, both for the spark-shell and compiled jars.

Thanks!

@costin
Copy link
Member

costin commented Mar 3, 2015

Cheers - closing the issue.

@costin costin closed this as completed Mar 3, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement :Spark v2.1.0.Beta4
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@costin @simonlei