You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When I query it with datafusion or just parquet::ParquetRecordBatchReaderBuilder, it errors with: "missing offset index"
Seems like it is skipping writing offset indices if page statistics are globally disabled?
I would expect, if it doesn't write offset indices then it shouldn't try to filter pages by statistics, also it should be documented that set_column_statistics_enabled doesn't override global settings in this way.
The text was updated successfully, but these errors were encountered:
* Test disabling page index statistics (#4587)
* Apply suggestions from code review
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
---------
Co-authored-by: Andrew Lamb <andrew@nerdnetworks.org>
If I write files with:
When I query it with datafusion or just
parquet::ParquetRecordBatchReaderBuilder
, it errors with: "missing offset index"Seems like it is skipping writing offset indices if page statistics are globally disabled?
I would expect, if it doesn't write offset indices then it shouldn't try to filter pages by statistics, also it should be documented that
set_column_statistics_enabled
doesn't override global settings in this way.The text was updated successfully, but these errors were encountered: