Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add missing filters argument in calls to pyarrow.parquet.Dataset #5348

Merged
merged 2 commits into from Aug 29, 2019

Conversation

rjzamora
Copy link
Member

Addressing the pyarrow filtering problem raised in #5346 by adding the filters argument to _determine_dataset_parts. Filtering seems to work fine when the pq.Dataset call includes this argument (for gather_statistics=True cases).

Also, the name test_filters was changed to test_filters_pyarrow, because the original name is now used to test the scenario discussed in #5346

cc @DenisRen @TomAugspurger

  • Tests added / passed
  • Passes black dask / flake8 dask

@TomAugspurger TomAugspurger merged commit 1162554 into dask:master Aug 29, 2019
@rjzamora rjzamora deleted the filter-fix branch September 6, 2019 14:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants