Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error when field has no values #24

Closed
witq opened this issue Nov 7, 2017 · 2 comments
Closed

Error when field has no values #24

witq opened this issue Nov 7, 2017 · 2 comments

Comments

@witq
Copy link

witq commented Nov 7, 2017

When an optional field has 0 values, the generated file seems unreadable by parquet-tools, the error I get is can not read class org.apache.parquet.format.PageHeader: Required field 'num_nulls' was not found in serialized data! Struct: DataPageHeaderV2(num_values:10, num_nulls:0, num_rows:10, encoding:PLAIN, definition_levels_byte_length:20, repetition_levels_byte_length:0).
When I forced the num_nulls value to be no less than 1, it started working. I'm not sure if this is an issue with the module or with the way I'm trying to use it, so I'm letting you know.

@witq witq changed the title Error when field has no values ( Error when field has no values Nov 7, 2017
@asmuth
Copy link
Contributor

asmuth commented Nov 7, 2017

uh oh. yes, this is a problem with the thriftjs library that we are already working around in a number of places (the problem is that if the value is a literal >0< thriftjs will not acutally encode the field but leave the tag out instead).

will add a regression case and fix for this.

@asmuth asmuth closed this as completed in 0cefbb9 Nov 13, 2017
asmuth added a commit that referenced this issue Nov 13, 2017
@asmuth
Copy link
Contributor

asmuth commented Nov 13, 2017

github automatically closed this issue because I pushed 86682e6. the fix is in 0.8.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants