Skip to content

[C++][Parquet] Be better to check null_count, distinct_count and nan_count are negative when reading files #50692

Description

@HuaHuaY

Describe the enhancement requested

In parquet.thrift, null_count, distinct_count, and nan_count are defined as optional i64, so it is possible to encounter a negative value (whether due to an incorrect file writer implementation, file corruption, or other reasons). We should check for negative values. However, the Parquet format does not specify whether to throw an error or treat such values ​​as "missing" when encountered; I lean towards the latter, as it avoids blocking the reading of other data.

nan_count is not supported now. We can check null_count and distinct_count first, and check nan_count when we implement #50689.

Component(s)

C++, Parquet

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions