Deprecate fastparquet #10743
Deprecate fastparquet #10743
Conversation
| } | ||
| marks = {(w, r): [skip_marks[w], skip_marks[r]] for w in backends for r in backends} | ||
|
|
||
| # Custom marks |
There was a problem hiding this comment.
This was unused.
crusaderky
force-pushed
the
fastparquet
branch
from
c60e894
to
b1c71c4
Compare
crusaderky
force-pushed
the
fastparquet
branch
from
b1c71c4
to
7a469bf
Compare
crusaderky
force-pushed
the
fastparquet
branch
from
7a469bf
to
28f771d
Compare
There was a problem hiding this comment.
lgtm
feel free to merge @hendrikmakait if you want to take a look as well, otherwise I'll merge tomorrow
| Parquet library to use. Defaults to 'auto', which uses ``pyarrow`` if | ||
| it is installed, and falls back to ``fastparquet`` otherwise. | ||
| it is installed, and falls back to the deprecated ``fastparquet`` otherwise. |
There was a problem hiding this comment.
nit: Should we at least mention somewhere that fastparquet does not support all functionality?
There was a problem hiding this comment.
Q: which functionality does it not support? You could have asked?
There was a problem hiding this comment.
@martindurant: I've merely referred to the comments about functionality not supported within Dask's read parquet that have been removed in this PR. See the removed lines above this comment for more information.
There was a problem hiding this comment.
OK I understand. You mean that fastparquet as a backend in dask does not any more support all dask functionality (although fastparquet itself does).
I am disappointed in how this issue (the whole PR) was handled without any notice to me, perhaps my comment above was not the right place for it.
Nevertheless, let me state for the record: that fastparquet has remained remarkably stable, small and performant for a long time, while keeping up with all upstream changes in pandas. I am sorry to see it dropped like this.
|
Merging this one, can do the doc change in a follow up if we want to |
|
thx @crusaderky |
Changes to
to_parquetandread_parquet:engine="fastparquet"engine="arrow"andengine="pyarrow-dataset"as aliases forengine="pyarrow"engine=<engine class>, which is required by dask_cudf.