You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If you have a row selection which skips the first data page, SerializedPageReader will error incorrectly.
To Reproduce
Create a ParquetRecordBatchStream using a RowFilter that skips the first page in any column.
Expected behavior
This should work.
Additional context
In GenericColumnReader::has_next, when num_buffered_value is 0, we call GenericColumnReader::read_new_page. In GenericRecordReader::skip_records we have to check if we are at the end of the column, this will always read the first data page. In cases where the selection skips that page, it will not be fetched so we get an error.
The text was updated successfully, but these errors were encountered:
Describe the bug
If you have a row selection which skips the first data page,
SerializedPageReader
will error incorrectly.To Reproduce
Create a
ParquetRecordBatchStream
using aRowFilter
that skips the first page in any column.Expected behavior
This should work.
Additional context
In
GenericColumnReader::has_next
, whennum_buffered_value
is 0, we callGenericColumnReader::read_new_page
. InGenericRecordReader::skip_records
we have to check if we are at the end of the column, this will always read the first data page. In cases where the selection skips that page, it will not be fetched so we get an error.The text was updated successfully, but these errors were encountered: