-
Notifications
You must be signed in to change notification settings - Fork 278
Open
Labels
bugSomething isn't workingSomething isn't working
Description
Description
When Spark's ConstantFolding optimizer rule is disabled, all-literal trunc() and date_trunc() calls reach the native engine, which crashes with:
Invalid input to function DateTrunc. Expected (PrimitiveArray<Date32>, Scalar) or (PrimitiveArray<Date32>, StringArray)Invalid input to function TimestampTrunc. Expected (PrimitiveArray<TimestampMicrosecondType>, Scalar, String) or (PrimitiveArray<TimestampMicrosecondType>, StringArray, String)
The native functions expect at least one array input but receive all scalars.
How to Reproduce
SET spark.sql.optimizer.excludedRules=org.apache.spark.sql.catalyst.optimizer.ConstantFolding;
SELECT trunc(date('2024-06-15'), 'year');
SELECT date_trunc('year', timestamp('2024-06-15 10:30:45'));Expected Behavior
The native engine should handle scalar inputs gracefully — either by computing the correct result or by falling back to Spark.
Affected Expressions
trunc()(DateTrunc)date_trunc()(TimestampTrunc)
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working