Skip to content

TIMESTAMP_NANO is not converted in InternalRecordWrapper, causing predicate evaluation failures #14973

@ayushtkn

Description

@ayushtkn

Apache Iceberg version

1.10.1 (latest release)

Query engine

None

Please describe the bug 🐞

Iceberg’s InternalRecordWrapper does not handle TIMESTAMP_NANO , which causes incorrect values to be returned during expression evaluation (e.g., joins and residual filtering).

Currently, InternalRecordWrapper.converter(Type) handles:

DATE
TIME
TIMESTAMP (microsecond precision)

But it does not handle TIMESTAMP_NANO.

As a result:
For TIMESTAMP_NANO fields, InternalRecordWrapper returns the original Java object (LocalDateTime or OffsetDateTime)

Iceberg expression evaluation (e.g. BoundReference.eval) expects epoch nanoseconds (Long)
This leads to runtime failures such as:

Caused by: java.lang.IllegalStateException: Not an instance of java.lang.Long: 2025-12-30T11:19:00.123456789 at org.apache.iceberg.data.GenericRecord.get(GenericRecord.java:138) at org.apache.iceberg.data.InternalRecordWrapper.get(InternalRecordWrapper.java:101) at org.apache.iceberg.Accessors$PositionAccessor.get(Accessors.java:71) at org.apache.iceberg.Accessors$PositionAccessor.get(Accessors.java:58) at org.apache.iceberg.expressions.BoundReference.eval(BoundReference.java:41) at org.apache.iceberg.expressions.Evaluator$EvalVisitor.isNull(Evaluator.java:86) at org.apache.iceberg.expressions.Evaluator$EvalVisitor.isNull(Evaluator.java:51) at org.apache.iceberg.expressions.ExpressionVisitors$BoundVisitor.predicate(ExpressionVisitors.java:298) at org.apache.iceberg.expressions.ExpressionVisitors.visitEvaluator(ExpressionVisitors.java:390) at org.apache.iceberg.expressions.ExpressionVisitors.visitEvaluator(ExpressionVisitors.java:402) at org.apache.iceberg.expressions.Evaluator$EvalVisitor.eval(Evaluator.java:56) at org.apache.iceberg.expressions.Evaluator.eval(Evaluator.java:48) at org.apache.iceberg.mr.mapreduce.AbstractIcebergRecordReader.lambda$applyResidualFiltering$0(AbstractIcebergRecordReader.java:98) at org.apache.iceberg.io.CloseableIterable$4.shouldKeep(CloseableIterable.java:129)

Willingness to contribute

  • I can contribute a fix for this bug independently
  • I would be willing to contribute a fix for this bug with guidance from the Iceberg community
  • I cannot contribute a fix for this bug at this time

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions