Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQL: Comparison beetween datetime field and string behaves inconsistently when used in filter vs. projection, grouping and ordering #77055

Open
Luegg opened this issue Aug 31, 2021 · 3 comments · May be fixed by #87810
Labels
:Analytics/SQL SQL querying >bug Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)

Comments

@Luegg
Copy link
Contributor

Luegg commented Aug 31, 2021

All kinds of comparisons between a datetime field and a string behave as expected in WHERE clauses but produce null in SELECT, GROUP BY and ORDER BY clauses.

A simple example is:

SELECT birth_date = '1955-10-04T00:00:00.000Z' a FROM test_emp WHERE a

       a
---------------
null

The query correctly produces one record but the a field should contain true.

Another example with GROUP BY and <:

SELECT COUNT(*) FROM test_emp GROUP BY birth_date < '1955-10-04T00:00:00.000Z'

   count(*)
---------------
100

Expected result contains two groups.

@Luegg Luegg added >bug :Analytics/SQL SQL querying labels Aug 31, 2021
@elasticmachine elasticmachine added the Team:QL (Deprecated) Meta label for query languages team label Aug 31, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-ql (Team:QL)

@costin
Copy link
Member

costin commented Aug 31, 2021

This is a good example for making date/math explicit, even if it's not exposed to the user.
Currently we delegate to ES however as we've discovered this can lead to some issues when things are being rewritten. In this issue, the problem lies with the evaluation (outside of ES) which fails.
One could argue that comparing a date time to a string should not be supported due to type compatibility. I tend to agree with that and I think it will expose the underlying behavior - that is promote the string into a date/time (or in case of a failure to promote into a detailed exception):

SELECT birth_date = '1955-10-04T00:00:00.000Z'  FROM ... // becomes
SELECT birthdate == CAST('1955-10-04T00:00:00.000Z' AS birthdate.type)  // OR
SELECT birth_date = '1955-10-04T00:00:00.000Z' :: DATETIME

Not easy to do since we need to take into account the date/math syntax however I'm hopeful we can reuse that code so we don't end up with duplicated work.

@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/SQL SQL querying >bug Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants