Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Excluding fields when writing JSON documents from Spark to Elasticsearch doesn't work #381

Closed
itaiy opened this issue Feb 19, 2015 · 1 comment
Closed

Excluding fields when writing JSON documents from Spark to Elasticsearch doesn't work #381

itaiy opened this issue Feb 19, 2015 · 1 comment
Labels
bug :Spark v2.1.0.Beta4

Comments

@itaiy
Copy link

itaiy commented Feb 19, 2015

When working with Spark and using RDD.saveJsonToEs(), setting es.mapping.exclude doesn't actually exclude the field from the final document.
I've used the latest elasticsearch-spark_2.10 artifact (version 2.1.0.BUILD-SNAPSHOT taken from http://oss.sonatype.org/content/repositories/snapshots).
I've placed a small test application to reproduce this issue in https://gist.github.com/itaiy/1067deddae09c16077f0
@costin
Copy link
Member

costin commented Apr 28, 2015

This has been fixed in master a while back - it will be part of the upcoming Beta4. Please try it out - if the problem persists, please raise a new issue.

Thanks!

@costin costin closed this as completed Apr 28, 2015
@armaseg armaseg mentioned this issue May 19, 2016
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug :Spark v2.1.0.Beta4
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@costin @itaiy