Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ignore _source field not mapped #648

Closed
costin opened this issue Jan 7, 2016 · 1 comment
Closed

Ignore _source field not mapped #648

costin opened this issue Jan 7, 2016 · 1 comment
Labels
enhancement :Rest :Spark v2.2.0

Comments

@costin
Copy link
Member

costin commented Jan 7, 2016

In case of explicit mappings, it is common for the _source to have parts of it ignored (unmapped). This can trip off parsing as the connector doesn't know what to do with it in case of a strict type schema (Spark SQL).
To be consistent across the board, by default the connector should look at the mapping and use that; the user can change this behaviour (basically parse all _source into objects if needed) but this is likely less common (and will not work with Spark SQL for example).
This was referenced Jan 7, 2016
Closed
Closed
Closed
costin added a commit that referenced this issue Jan 19, 2016
@costin
[REST] By default filter non-mapped fields
13c3422 
Additionally generalize the es.read.field mechanism to filter received
data (in addition to the mapping)

relates #648 #649
costin added a commit that referenced this issue Jan 20, 2016
@costin
[REST] By default filter non-mapped fields
5f8496c 
Additionally generalize the es.read.field mechanism to filter received
data (in addition to the mapping)

relates #648 #649
@costin costin added v2.2.0 and removed v2.3.0 labels Jan 24, 2016
@costin
Copy link
Member Author

costin commented Jan 24, 2016

Available in master/2.2

@costin costin closed this as completed Jan 24, 2016
@anopheles anopheles mentioned this issue Oct 6, 2016
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement :Rest :Spark v2.2.0
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@costin