-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Closed
Labels
Description
- Function calls with literal-only operands get evaluated in the broker during query planning / optimization itself instead of during query execution in the servers as an optimization.
- This is done through the PinotEvaluateLiteralRule.
- However, there's a bug that prevents
CASTfunctions from being evaluated. TheCASToperator in Calcite only has one operand (the value to be cast) and the type to be cast to is determined by the operator's return type. Pinot'sCASTfunction implementation requires two arguments: the value to be cast and the target type. - This discrepancy causes the
CASTfunction to not be found in the rule implementation, and the function call is retained as is. - This can cause a myriad of downstream issues. For instance, take a query like:
select AirlineID from airlineStats where ts > '2000-05-16T00:00:00' and ts < '2025-05-16T00:00:00';
- The filter becomes
AND(>($82, CAST(_UTF-8'2000-05-16T00:00:00'):TIMESTAMP(0) NOT NULL), <($82, CAST(_UTF-8'2025-05-16T00:00:00'):TIMESTAMP(0) NOT NULL))where the casts are added by Calcites to convert the string literals to the appropriate type. - Since the
CASTdoesn't get evaluated during query planning, it makes it down all the way to the leaf stage, where PredicateComparisonRewriter rewrites the filter expression to:
FilterAnd
FilterExpression(predicate=[minus(ts,cast('2000-05-16T00:00:00','TIMESTAMP')) > '0'], operator=[RANGE])
FilterExpression(predicate=[minus(ts,cast('2025-05-16T00:00:00','TIMESTAMP')) < '0'], operator=[RANGE])
- This is a big issue because it'll prevent the filter from using range indexes.
Reactions are currently unavailable