Skip to content

CAST function calls with literal-only operands don't get evaluated during query planning in the MSQE #16419

@yashmayya

Description

@yashmayya
  • Function calls with literal-only operands get evaluated in the broker during query planning / optimization itself instead of during query execution in the servers as an optimization.
  • This is done through the PinotEvaluateLiteralRule.
  • However, there's a bug that prevents CAST functions from being evaluated. The CAST operator in Calcite only has one operand (the value to be cast) and the type to be cast to is determined by the operator's return type. Pinot's CAST function implementation requires two arguments: the value to be cast and the target type.
  • This discrepancy causes the CAST function to not be found in the rule implementation, and the function call is retained as is.
  • This can cause a myriad of downstream issues. For instance, take a query like:
select AirlineID from airlineStats where ts > '2000-05-16T00:00:00' and ts < '2025-05-16T00:00:00';
  • The filter becomes AND(>($82, CAST(_UTF-8'2000-05-16T00:00:00'):TIMESTAMP(0) NOT NULL), <($82, CAST(_UTF-8'2025-05-16T00:00:00'):TIMESTAMP(0) NOT NULL)) where the casts are added by Calcites to convert the string literals to the appropriate type.
  • Since the CAST doesn't get evaluated during query planning, it makes it down all the way to the leaf stage, where PredicateComparisonRewriter rewrites the filter expression to:
FilterAnd
  FilterExpression(predicate=[minus(ts,cast('2000-05-16T00:00:00','TIMESTAMP')) > '0'], operator=[RANGE])
  FilterExpression(predicate=[minus(ts,cast('2025-05-16T00:00:00','TIMESTAMP')) < '0'], operator=[RANGE])
  • This is a big issue because it'll prevent the filter from using range indexes.

Metadata

Metadata

Assignees

Labels

bugmulti-stageRelated to the multi-stage query engine

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions