Parquet: Fix NPE in value reader building for projections#1164
Parquet: Fix NPE in value reader building for projections#1164rdblue merged 3 commits intoapache:masterfrom
Conversation
93b8f5e to
775c74d
Compare
|
I was trying to look into why this was only a failure for Parquet. I might be wrong here, but It seems that for ORC and Avro were are building the readerFuncs in |
|
Also should this be addressed for other formats? |
|
@rdsr, the test applies to all formats and only Parquet was broken. |
The equivalent Avro builder doesn't use the expected primitive at all. The variable name is |
I believe it is not getting caught because in ORC (as well as Avro) the reader func are utilizing the read schema which completely match the Iceberg expected schema so there will not be a case where the iceberg type is null and the format type is not, whereas in Parquet the reader func is using the file schema. |
Co-authored-by: 范欣欣 <hzfanxinxin@corp.netease.com>
The visitor that builds Parquet value readers for Iceberg generics did not handle cases where the expected type was null because a field was is not projected. This was not a problem for most primitive types that did not use the expected type, but cases that did would throw a
NullPointerException.Fixes #1117.