Skip to content

Native engine crashes on literal DateTrunc and TimestampTrunc #3342

@andygrove

Description

@andygrove

Description

When Spark's ConstantFolding optimizer rule is disabled, all-literal trunc() and date_trunc() calls reach the native engine, which crashes with:

  • Invalid input to function DateTrunc. Expected (PrimitiveArray<Date32>, Scalar) or (PrimitiveArray<Date32>, StringArray)
  • Invalid input to function TimestampTrunc. Expected (PrimitiveArray<TimestampMicrosecondType>, Scalar, String) or (PrimitiveArray<TimestampMicrosecondType>, StringArray, String)

The native functions expect at least one array input but receive all scalars.

How to Reproduce

SET spark.sql.optimizer.excludedRules=org.apache.spark.sql.catalyst.optimizer.ConstantFolding;
SELECT trunc(date('2024-06-15'), 'year');
SELECT date_trunc('year', timestamp('2024-06-15 10:30:45'));

Expected Behavior

The native engine should handle scalar inputs gracefully — either by computing the correct result or by falling back to Spark.

Affected Expressions

  • trunc() (DateTrunc)
  • date_trunc() (TimestampTrunc)

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions