Skip to content

Updates for fastparquet evolution#9650

Merged
jrbourbeau merged 8 commits intodask:mainfrom
martindurant:fp_update
Nov 16, 2022
Merged

Updates for fastparquet evolution#9650
jrbourbeau merged 8 commits intodask:mainfrom
martindurant:fp_update

Conversation

@martindurant
Copy link
Copy Markdown
Member

@martindurant martindurant commented Nov 11, 2022

  • Closes #xxxx
  • Tests added / passed
  • Passes pre-commit run --all-files

Makes dask/fastparquet#818 pass

Copy link
Copy Markdown
Contributor

@phobson phobson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, @martindurant

@martindurant
Copy link
Copy Markdown
Member Author

@rjzamora : a quick and simple one, if you have a moment

Copy link
Copy Markdown
Member

@rjzamora rjzamora left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor comment fix. Otherwise, LGTM

Co-authored-by: Richard (Rick) Zamora <rzamora217@gmail.com>
Copy link
Copy Markdown
Member

@jrbourbeau jrbourbeau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @martindurant. I think we want to trigger an upstream CI build by including a commit with test-upstream in it so we can make sure fastparquet_version > parse_version("0.8.3") is True (note you'll also want to merge main as there were some changes to fix the upstream build that were just merged)

@martindurant
Copy link
Copy Markdown
Member Author

There is no release of fastparquet yet, since dask/fastparquet#818 is waiting on this PR

@jrbourbeau
Copy link
Copy Markdown
Member

The upstream build will install fastparquet directly from main. I assumed this was using some new feature that's not yet released, but in fastparquets main branch -- is that not the case?

@martindurant
Copy link
Copy Markdown
Member Author

No, I didn't even merge it; but I will now, just to ease the impasse :).
fastparquet also tests against main, of course!

@martindurant
Copy link
Copy Markdown
Member Author

done

@martindurant
Copy link
Copy Markdown
Member Author

dask/array/tests/test_stats.py::test_two[ttest_1samp-kwargs2] is nothing to do with this PR

@jrbourbeau jrbourbeau changed the title Updates for fastparquet evolution Updates for fastparquet evolution Nov 16, 2022
@jrbourbeau jrbourbeau merged commit 6a02bcb into dask:main Nov 16, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants