Apache Iceberg version
1.10.1 (latest release)
Query engine
None
Please describe the bug 🐞
Iceberg’s InternalRecordWrapper does not handle TIMESTAMP_NANO , which causes incorrect values to be returned during expression evaluation (e.g., joins and residual filtering).
Currently, InternalRecordWrapper.converter(Type) handles:
DATE
TIME
TIMESTAMP (microsecond precision)
But it does not handle TIMESTAMP_NANO.
As a result:
For TIMESTAMP_NANO fields, InternalRecordWrapper returns the original Java object (LocalDateTime or OffsetDateTime)
Iceberg expression evaluation (e.g. BoundReference.eval) expects epoch nanoseconds (Long)
This leads to runtime failures such as:
Caused by: java.lang.IllegalStateException: Not an instance of java.lang.Long: 2025-12-30T11:19:00.123456789 at org.apache.iceberg.data.GenericRecord.get(GenericRecord.java:138) at org.apache.iceberg.data.InternalRecordWrapper.get(InternalRecordWrapper.java:101) at org.apache.iceberg.Accessors$PositionAccessor.get(Accessors.java:71) at org.apache.iceberg.Accessors$PositionAccessor.get(Accessors.java:58) at org.apache.iceberg.expressions.BoundReference.eval(BoundReference.java:41) at org.apache.iceberg.expressions.Evaluator$EvalVisitor.isNull(Evaluator.java:86) at org.apache.iceberg.expressions.Evaluator$EvalVisitor.isNull(Evaluator.java:51) at org.apache.iceberg.expressions.ExpressionVisitors$BoundVisitor.predicate(ExpressionVisitors.java:298) at org.apache.iceberg.expressions.ExpressionVisitors.visitEvaluator(ExpressionVisitors.java:390) at org.apache.iceberg.expressions.ExpressionVisitors.visitEvaluator(ExpressionVisitors.java:402) at org.apache.iceberg.expressions.Evaluator$EvalVisitor.eval(Evaluator.java:56) at org.apache.iceberg.expressions.Evaluator.eval(Evaluator.java:48) at org.apache.iceberg.mr.mapreduce.AbstractIcebergRecordReader.lambda$applyResidualFiltering$0(AbstractIcebergRecordReader.java:98) at org.apache.iceberg.io.CloseableIterable$4.shouldKeep(CloseableIterable.java:129)
Willingness to contribute
Apache Iceberg version
1.10.1 (latest release)
Query engine
None
Please describe the bug 🐞
Iceberg’s InternalRecordWrapper does not handle
TIMESTAMP_NANO, which causes incorrect values to be returned during expression evaluation (e.g., joins and residual filtering).Currently,
InternalRecordWrapper.converter(Type)handles:But it does not handle
TIMESTAMP_NANO.As a result:
For
TIMESTAMP_NANOfields,InternalRecordWrapperreturns the original Java object(LocalDateTime or OffsetDateTime)Iceberg expression evaluation (e.g.
BoundReference.eval) expects epoch nanoseconds (Long)This leads to runtime failures such as:
Caused by: java.lang.IllegalStateException: Not an instance of java.lang.Long: 2025-12-30T11:19:00.123456789 at org.apache.iceberg.data.GenericRecord.get(GenericRecord.java:138) at org.apache.iceberg.data.InternalRecordWrapper.get(InternalRecordWrapper.java:101) at org.apache.iceberg.Accessors$PositionAccessor.get(Accessors.java:71) at org.apache.iceberg.Accessors$PositionAccessor.get(Accessors.java:58) at org.apache.iceberg.expressions.BoundReference.eval(BoundReference.java:41) at org.apache.iceberg.expressions.Evaluator$EvalVisitor.isNull(Evaluator.java:86) at org.apache.iceberg.expressions.Evaluator$EvalVisitor.isNull(Evaluator.java:51) at org.apache.iceberg.expressions.ExpressionVisitors$BoundVisitor.predicate(ExpressionVisitors.java:298) at org.apache.iceberg.expressions.ExpressionVisitors.visitEvaluator(ExpressionVisitors.java:390) at org.apache.iceberg.expressions.ExpressionVisitors.visitEvaluator(ExpressionVisitors.java:402) at org.apache.iceberg.expressions.Evaluator$EvalVisitor.eval(Evaluator.java:56) at org.apache.iceberg.expressions.Evaluator.eval(Evaluator.java:48) at org.apache.iceberg.mr.mapreduce.AbstractIcebergRecordReader.lambda$applyResidualFiltering$0(AbstractIcebergRecordReader.java:98) at org.apache.iceberg.io.CloseableIterable$4.shouldKeep(CloseableIterable.java:129)Willingness to contribute