Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SQL: HAVING with functions that are not covered by regular metric aggregations in ES doesn't work #33519

Open
astefan opened this issue Sep 7, 2018 · 5 comments
Labels
:Analytics/SQL SQL querying >bug Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)

Comments

@astefan
Copy link
Contributor

astefan commented Sep 7, 2018

sql> select avg("salary"), day("hire_date") from "test_emp" group by day("hire_date") having day("hire_date") = 1;
Bad request [Found 1 problem(s)
line 1:89: Cannot filter HAVING on non-aggregate [hire_date]; consider using WHERE instead]

A query like this one is normally translated into ES DSL more or less like this:

{
    "size": 0,
    "_source": false,
    "stored_fields": "_none_",
    "aggregations": {
        "groupby": {
            "composite": {
                "size": 1000,
                "sources": [
                    {
                        "493": {
                            "terms": {
                                "script": {
                                    "source": "InternalSqlScriptUtils.dateTimeChrono(doc[params.v0].value.millis, params.v1, params.v2)",
                                    "lang": "painless",
                                    "params": {
                                        "v0": "hire_date",
                                        "v1": "UTC",
                                        "v2": "DAY_OF_MONTH"
                                    }
                                },
                                "missing_bucket": true,
                                "value_type": "number",
                                "order": "asc"
                            }
                        }
                    }
                ]
            },
            "aggregations": {
                "494": {
                    "avg": {                              <------------
                        "field": "salary"              <------------ the buckets path used in the condition
                    }                                        <------------
                },
                "497": {
                    "bucket_selector": {
                        "buckets_path": {
                            "a0": "494"
                        },
                        "script": {
                            "source": "params.a0 = params.v0",
                            "lang": "painless",
                            "params": {
                                "v0": 1
                            }
                        },
                        "gap_policy": "skip"
                    }
                }
            }
        }
    }
}

While ES does offer avg, min, max etc metric aggregations, we would need the key of parent aggregation (the composite one) for the bucket_selector script, which I am not sure it's possible.

@astefan astefan added >bug :Analytics/SQL SQL querying labels Sep 7, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search-aggs

@costin
Copy link
Member

costin commented Sep 7, 2018

I think the error is correct; HAVING is used for aggregates only not fields (whether grouped or not).
Converting HAVING to WHERE should fix the error and the query should work.

@astefan
Copy link
Contributor Author

astefan commented Sep 10, 2018

That's fair @costin.
How about the following use case?

sql> select dayname("hire_date") from "test_emp" group by dayname("hire_date") having max(emp_no) > ASCII(dayname("hire_date"));
Bad request [Found 1 problem(s)
line 1:82: Cannot filter HAVING on non-aggregate [hire_date]; consider using WHERE instead]

@matriv
Copy link
Contributor

matriv commented Mar 30, 2020

@elastic/es-ql

@rjernst rjernst added the Team:QL (Deprecated) Meta label for query languages team label May 4, 2020
@wchaparro wchaparro removed the Team:QL (Deprecated) Meta label for query languages team label Jan 17, 2024
@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Jan 17, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/SQL SQL querying >bug Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)
Projects
None yet
Development

No branches or pull requests

7 participants