You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When an optional field has 0 values, the generated file seems unreadable by parquet-tools, the error I get is can not read class org.apache.parquet.format.PageHeader: Required field 'num_nulls' was not found in serialized data! Struct: DataPageHeaderV2(num_values:10, num_nulls:0, num_rows:10, encoding:PLAIN, definition_levels_byte_length:20, repetition_levels_byte_length:0).
When I forced the num_nulls value to be no less than 1, it started working. I'm not sure if this is an issue with the module or with the way I'm trying to use it, so I'm letting you know.
The text was updated successfully, but these errors were encountered:
witq
changed the title
Error when field has no values (
Error when field has no values
Nov 7, 2017
uh oh. yes, this is a problem with the thriftjs library that we are already working around in a number of places (the problem is that if the value is a literal >0< thriftjs will not acutally encode the field but leave the tag out instead).
When an optional field has 0 values, the generated file seems unreadable by parquet-tools, the error I get is
can not read class org.apache.parquet.format.PageHeader: Required field 'num_nulls' was not found in serialized data! Struct: DataPageHeaderV2(num_values:10, num_nulls:0, num_rows:10, encoding:PLAIN, definition_levels_byte_length:20, repetition_levels_byte_length:0)
.When I forced the
num_nulls
value to be no less than 1, it started working. I'm not sure if this is an issue with the module or with the way I'm trying to use it, so I'm letting you know.The text was updated successfully, but these errors were encountered: