Describe the enhancement requested
In parquet.thrift, null_count, distinct_count, and nan_count are defined as optional i64, so it is possible to encounter a negative value (whether due to an incorrect file writer implementation, file corruption, or other reasons). We should check for negative values. However, the Parquet format does not specify whether to throw an error or treat such values as "missing" when encountered; I lean towards the latter, as it avoids blocking the reading of other data.
nan_count is not supported now. We can check null_count and distinct_count first, and check nan_count when we implement #50689.
Component(s)
C++, Parquet
Describe the enhancement requested
In
parquet.thrift,null_count,distinct_count, andnan_countare defined asoptional i64, so it is possible to encounter a negative value (whether due to an incorrect file writer implementation, file corruption, or other reasons). We should check for negative values. However, the Parquet format does not specify whether to throw an error or treat such values as "missing" when encountered; I lean towards the latter, as it avoids blocking the reading of other data.nan_countis not supported now. We can checknull_countanddistinct_countfirst, and checknan_countwhen we implement #50689.Component(s)
C++, Parquet