update ARRAY_OVERLAP to plan with ArrayContainsElement for ARRAY columns #15451
Conversation
| if (simpleExtractionExpr.isDirectColumnAccess() && simpleExtractionExpr.isArray()) { | ||
| // To convert this expression filter into an OR of ArrayContainsElement filters, we need to extract all array | ||
| // elements. | ||
| if (expr.isLiteral()) { |
There was a problem hiding this comment.
what will happen if its an empty array?
There was a problem hiding this comment.
hmm, so i guess it will fail a precondition check when it makes the OR filter because fields will be empty...
However, it currently isn't really possible to make an empty array literal because ARRAY[] isn't accepted by calcite (or the native druid array expression), so I'm not really sure how important it is to handle this case either.
I can update this (and ArrayContainsOperatorConversion) to return null in this case since we don't really handle it, or true i guess would also probably be correct?
| @@ -149,6 +153,42 @@ public DimFilter toDruidFilter( | |||
| ); | |||
| } | |||
| } | |||
|
|
|||
| // if the input is a direct array column, we can use sweet array filter | |||
There was a problem hiding this comment.
nit
| // if the input is a direct array column, we can use sweet array filter | |
| // if the input is a direct array column, we can use sweet array contains filter |
There was a problem hiding this comment.
heh, i just copied this section from ArrayContainsOperatorConversion and switched AND to OR
…mns (apache#15451) Updates ARRAY_OVERLAP to use the same ArrayContainsElement filter added in apache#15366 when filtering ARRAY typed columns so that it can also use indexes like ARRAY_CONTAINS.
…mns (apache#15451) Updates ARRAY_OVERLAP to use the same ArrayContainsElement filter added in apache#15366 when filtering ARRAY typed columns so that it can also use indexes like ARRAY_CONTAINS.
Description
Updates
ARRAY_OVERLAPto use the sameArrayContainsElementfilter added in #15366 when filtering ARRAY typed columns so that it can also use indexes likeARRAY_CONTAINS.For query:
After:
This PR has: