New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ESQL: PushdownRegexExtract doesn't take into account shadow attributes #105434
Labels
:Analytics/ES|QL
AKA ESQL
>bug
Team:Analytics
Meta label for analytical engine team (ESQL/Aggs/Geo)
v8.14.0
Comments
elasticsearchmachine
added
the
Team:Analytics
Meta label for analytical engine team (ESQL/Aggs/Geo)
label
Feb 13, 2024
Pinging @elastic/es-analytical-engine (Team:Analytics) |
alex-spies
changed the title
ESQL: PushdownRegexEval doesn't take into account shadow attributes
ESQL: PushdownRegexExtract doesn't take into account shadow attributes
Feb 15, 2024
I have reproducers for both that result in NPEs if run as CSV tests:
|
elasticsearchmachine
pushed a commit
that referenced
this issue
Feb 26, 2024
Fix #105434 Fixes accidental shadowing when pushing down `GROK`/`DISSECT`, `EVAL` or `ENRICH` past a `SORT`. Example for how this works: ``` ... | SORT x | EVAL x = y ... pushing this down just like that would be incorrect as x is used in the SORT, so we turn this essentially into ... | EVAL $$x = x | EVAL x = y | SORT $$x | DROP $$x ... ``` The same logic is applied to `GROK`/`DISSECT` and `ENRICH`. This allows to re-enable the dependency checker (after fixing a small bug in it when handling `ENRICH`).
alex-spies
added a commit
to alex-spies/elasticsearch
that referenced
this issue
Feb 26, 2024
Fix elastic#105434 Fixes accidental shadowing when pushing down `GROK`/`DISSECT`, `EVAL` or `ENRICH` past a `SORT`. Example for how this works: ``` ... | SORT x | EVAL x = y ... pushing this down just like that would be incorrect as x is used in the SORT, so we turn this essentially into ... | EVAL $$x = x | EVAL x = y | SORT $$x | DROP $$x ... ``` The same logic is applied to `GROK`/`DISSECT` and `ENRICH`. This allows to re-enable the dependency checker (after fixing a small bug in it when handling `ENRICH`).
elasticsearchmachine
pushed a commit
that referenced
this issue
Feb 26, 2024
#105808) * ESQL: Fix wrong attribute shadowing in pushdown rules (#105650) Fix #105434 Fixes accidental shadowing when pushing down `GROK`/`DISSECT`, `EVAL` or `ENRICH` past a `SORT`. Example for how this works: ``` ... | SORT x | EVAL x = y ... pushing this down just like that would be incorrect as x is used in the SORT, so we turn this essentially into ... | EVAL $$x = x | EVAL x = y | SORT $$x | DROP $$x ... ``` The same logic is applied to `GROK`/`DISSECT` and `ENRICH`. This allows to re-enable the dependency checker (after fixing a small bug in it when handling `ENRICH`). * Make OptimizerRules compile again
fang-xing-esql
pushed a commit
to fang-xing-esql/Elasticsearch
that referenced
this issue
Mar 8, 2024
Fix elastic#105434 Fixes accidental shadowing when pushing down `GROK`/`DISSECT`, `EVAL` or `ENRICH` past a `SORT`. Example for how this works: ``` ... | SORT x | EVAL x = y ... pushing this down just like that would be incorrect as x is used in the SORT, so we turn this essentially into ... | EVAL $$x = x | EVAL x = y | SORT $$x | DROP $$x ... ``` The same logic is applied to `GROK`/`DISSECT` and `ENRICH`. This allows to re-enable the dependency checker (after fixing a small bug in it when handling `ENRICH`).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
:Analytics/ES|QL
AKA ESQL
>bug
Team:Analytics
Meta label for analytical engine team (ESQL/Aggs/Geo)
v8.14.0
Description
Grok and Dissect produce attributes which can clash with existing fields creating new attributes (with the same name).
When reordering this through the tree however, the shadow attributes are not considered leading to inconsistencies in the tree:
The text was updated successfully, but these errors were encountered: