Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SQL: Perform lazy evaluation of mismatched mappings #35659

Closed
costin opened this issue Nov 16, 2018 · 3 comments
Closed

SQL: Perform lazy evaluation of mismatched mappings #35659

costin opened this issue Nov 16, 2018 · 3 comments
Labels
:Analytics/SQL SQL querying >enhancement v6.6.0 v7.0.0-beta1

Comments

@costin
Copy link
Member

costin commented Nov 16, 2018

Especially in the logging case, it is common to have mappings with some inconsistencies yet these fields are never used in the result, e.g.:

SELECT longitude, latitude, dateTime 
FROM "mike_location*"
ORDER BY dateTime
LIMIT 1

returns

Unexpected error from Elasticsearch: [verification_exception] Found 1 problem(s) line 2:6: [mike_location*] points to indices with incompatible mappings; field [previous_location] is mapped in [2] different ways: [object] in [mike_location], [geo_point] in [mike_location-v2]

While the message is correct, note that the problematic fields are not used.
Such queries should be allowed since the problematic fields are never used and thus can be safely ignored.
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search-aggs

@costin
Copy link
Member Author

costin commented Nov 16, 2018

/cc @alexfrancoeur

@alexfrancoeur
Copy link

cc: @elastic/kibana-canvas

costin added a commit to costin/elasticsearch that referenced this issue Nov 17, 2018
@costin
SQL: Perform lazy evaluation of mismatched mappings
6fc1947 
Move away from performing eager, fail-fast validation of mismatched
 mapping to a lazy evaluation based on the fields actually used in the
 query. This allows queries to run on the parts of the indices that
 "work" which is not just convenient but also a necessity for large
 mappings (like logging) where alignment is hard/impossible to achieve.

Fix elastic#35659
@costin costin mentioned this issue Nov 17, 2018
Merged
costin added a commit to costin/elasticsearch that referenced this issue Nov 18, 2018
@costin
SQL: Perform lazy evaluation of mismatched mappings
2a9f8c7 
Move away from performing eager, fail-fast validation of mismatched
 mapping to a lazy evaluation based on the fields actually used in the
 query. This allows queries to run on the parts of the indices that
 "work" which is not just convenient but also a necessity for large
 mappings (like logging) where alignment is hard/impossible to achieve.

Fix elastic#35659
costin added a commit to costin/elasticsearch that referenced this issue Nov 18, 2018
@costin
SQL: Perform lazy evaluation of mismatched mappings
11dea38 
Move away from performing eager, fail-fast validation of mismatched
 mapping to a lazy evaluation based on the fields actually used in the
 query. This allows queries to run on the parts of the indices that
 "work" which is not just convenient but also a necessity for large
 mappings (like logging) where alignment is hard/impossible to achieve.

Fix elastic#35659
costin added a commit that referenced this issue Nov 21, 2018
@costin
SQL: Perform lazy evaluation of mismatched mappings (#35676)
03f0037 
Move away from performing eager, fail-fast validation of mismatched
 mapping to a lazy evaluation based on the fields actually used in the
 query. This allows queries to run on the parts of the indices that
 "work" which is not just convenient but also a necessity for large
 mappings (like logging) where alignment is hard/impossible to achieve.

Fix #35659
costin added a commit that referenced this issue Nov 21, 2018
@costin
SQL: Perform lazy evaluation of mismatched mappings (#35676)
eff8262 
Move away from performing eager, fail-fast validation of mismatched
 mapping to a lazy evaluation based on the fields actually used in the
 query. This allows queries to run on the parts of the indices that
 "work" which is not just convenient but also a necessity for large
 mappings (like logging) where alignment is hard/impossible to achieve.

Fix #35659

(cherry picked from commit 03f0037)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
:Analytics/SQL SQL querying >enhancement v6.6.0 v7.0.0-beta1
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@costin @colings86 @elasticmachine @alexfrancoeur