Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot decode dictionary of type INT96 when reading imported Spark parquet table #8990

Closed
manuzhang opened this issue Nov 6, 2023 · 0 comments · Fixed by #8988
Closed

Cannot decode dictionary of type INT96 when reading imported Spark parquet table #8990

manuzhang opened this issue Nov 6, 2023 · 0 comments · Fixed by #8988

Comments

@manuzhang
Copy link
Contributor

manuzhang commented Nov 6, 2023

Apache Iceberg version

1.2.1

Query engine

Spark

Please describe the bug 🐞

The following exception was thrown when reading timestamp column from an imported Spark parquet table with filter.

java.lang.IllegalArgumentException: Cannot decode dictionary of type: INT96
    at org.apache.iceberg.parquet.ParquetDictionaryRowGroupFilter$EvalVisitor.dict(ParquetDictionaryRowGroupFilter.java:458)
    at org.apache.iceberg.parquet.ParquetDictionaryRowGroupFilter$EvalVisitor.eq(ParquetDictionaryRowGroupFilter.java:293)
    at org.apache.iceberg.parquet.ParquetDictionaryRowGroupFilter$EvalVisitor.eq(ParquetDictionaryRowGroupFilter.java:80)
    at org.apache.iceberg.expressions.ExpressionVisitors$BoundExpressionVisitor.predicate(ExpressionVisitors.java:162)
    at org.apache.iceberg.expressions.ExpressionVisitors.visitEvaluator(ExpressionVisitors.java:390)
    at org.apache.iceberg.expressions.ExpressionVisitors.visitEvaluator(ExpressionVisitors.java:409)
    at org.apache.iceberg.parquet.ParquetDictionaryRowGroupFilter$EvalVisitor.eval(ParquetDictionaryRowGroupFilter.java:118)
    at org.apache.iceberg.parquet.ParquetDictionaryRowGroupFilter$EvalVisitor.access$100(ParquetDictionaryRowGroupFilter.java:80)
    at org.apache.iceberg.parquet.ParquetDictionaryRowGroupFilter.shouldRead(ParquetDictionaryRowGroupFilter.java:74)
    at org.apache.iceberg.parquet.ReadConf.<init>(ReadConf.java:119)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant