Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hang due to infinite loop when reading some parquet files with RLE encoding and bit packing #1458

Closed
anliakho2 opened this issue Mar 17, 2022 · 1 comment · Fixed by #1555
Labels
bug parquet Changes to the parquet crate

Comments

@anliakho2
Copy link

Describe the bug
Some parquet writers produce RLE encoded dictionaries with bit packing even if value count is not divisible by 8.
This violates the assumption decoder makes about the number of values present in the bit-packed run, causing infinite loop and hang.

To Reproduce
Attempt to read parquet file where number of values is not power of 2 but RLE encoding with bit packing was used.

Expected behavior
File should be read correctly.

Additional context

@alamb
Copy link
Contributor

alamb commented Mar 17, 2022

Thank you for the report @anliakho2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug parquet Changes to the parquet crate
Projects
None yet
2 participants