Improve messaging around sorted parquet columns for index #5265
Conversation
dask/dataframe/io/parquet/core.py
Outdated
| "Multiple sorted columns found, cannot autodetect index", | ||
| "Multiple sorted columns found %s, cannot\n " | ||
| "autodetect index. Will continue without an index.\n" | ||
| "To pick an index column, use the index= kwargs; to \n" |
There was a problem hiding this comment.
Change "index=kwargs" to index keyword argument? Or at least make clear that it's a single argument, right?
| pd.DataFrame({"cola": range(10), "colb": range(10)}), npartitions=2 | ||
| ) | ||
| df.to_parquet(tmpdir, engine=engine, write_index=False) | ||
| with pytest.warns(RuntimeWarning) as record: |
There was a problem hiding this comment.
FYI, you can use match="'cola', 'colb'" now I think, rather than the as record stuff.
I'm not sure whether it asserts that there is just one assertion. But why do we assert that?
There was a problem hiding this comment.
There are other warnings too for pyarrow, so I think it needs to be this way
There was a problem hiding this comment.
I don't think so. I made a small test
import pytest
import warnings
def f(x):
warnings.warn("test", FutureWarning)
warnings.warn("user", UserWarning)
def test_foo():
with pytest.warns(UserWarning, match="user"):
f(0)and things seem fine with the match syntax.
There was a problem hiding this comment.
What I mean it, I wanted to capture the recorder object, to explicitly count the number of warnings, and make sure that supplying the kwarg makes that number go down, but doesn't go to zero.
|
Thanks! |
black dask/flake8 daskFixes #5260