Skip to content

Fix parquet filtering bug for "pyarrow-dataset" with pyarrow-3.0.0#7200

Merged
martindurant merged 4 commits intodask:masterfrom
rjzamora:dataset-filter-fix
Feb 19, 2021
Merged

Fix parquet filtering bug for "pyarrow-dataset" with pyarrow-3.0.0#7200
martindurant merged 4 commits intodask:masterfrom
rjzamora:dataset-filter-fix

Conversation

@rjzamora
Copy link
Copy Markdown
Member

@rjzamora rjzamora commented Feb 9, 2021

Fixes the bug reported in this 6376 comment. The problem only shows up (with the latest pyarrow-3.0.0 release) when filtering a non-partitioned column with split_row_groups=False.

@martindurant martindurant merged commit 1c9a962 into dask:master Feb 19, 2021
@rjzamora rjzamora deleted the dataset-filter-fix branch February 19, 2021 15:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants