-
Notifications
You must be signed in to change notification settings - Fork 45
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
TST: fix CI (black, dask master) #47
Conversation
I am thinking whether we should pin the version of black as we have it in geopandas or not. I am not sure about that, it can easily remain pinned for years unnoticed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should fix the last error on CI
@jakirkham I did catch those as well but there seems to be a bit more work apparently :D. |
Ok will keep an eye on it ;) Interestingly I can't mark my own comments as resolved (sorry about that) |
dask_geopandas/backends.py
Outdated
DASK_2021_06 = str(dask.__version__) > LooseVersion("2021.05.0") | ||
|
||
if DASK_2021_06: | ||
from dask.dataframe.utils import make_meta_obj as make_meta |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With latest dask, make_meta_obj
is a dispatch to handle meta creation for arbitrary object (for ex: object
, int
, dict
..). make_meta_obj
is being used to register against geopandas.GeoSeries
& geopandas.GeoDataFrame
- This will lead to incorrect behavior which we are seeing now in CI. Instead it has to be something like:
from dask.dataframe.utils import make_meta, make_meta_obj
@make_meta.register((geopandas.GeoSeries, geopandas.GeoDataFrame))
def make_meta_geodataframe(df, index=None):
return df.head(0)
@make_meta_obj.register((geopandas.GeoSeries, geopandas.GeoDataFrame, <any other objects of geopandas type>,...)):
def make_meta_object(x, index=None):
if is_arraylike(x) and x.shape:
return x[:0]
if index is not None:
index = make_meta(index)
if isinstance(x, dict):
return pd.DataFrame(
{c: _empty_series(c, d, index=index) for (c, d) in x.items()}, index=index
)
if isinstance(x, tuple) and len(x) == 2:
return _empty_series(x[0], x[1], index=index)
elif isinstance(x, (list, tuple)):
if not all(isinstance(i, tuple) and len(i) == 2 for i in x):
raise ValueError(
"Expected iterable of tuples of (name, dtype), got {0}".format(x)
)
return pd.DataFrame(
{c: _empty_series(c, d, index=index) for (c, d) in x},
columns=[c for c, d in x],
index=index,
)
elif not hasattr(x, "dtype") and x is not None:
# could be a string, a dtype object, or a python type. Skip `None`,
# because it is implictly converted to `dtype('f8')`, which we don't
# want here.
try:
dtype = np.dtype(x)
return _scalar_from_dtype(dtype)
except Exception:
# Continue on to next check
pass
if is_scalar(x):
return _nonempty_scalar(x)
raise TypeError("Don't know how to create metadata from {0}".format(x))
Thanks @galipremsagar! I have fixed |
Thanks for the help @jakirkham & @galipremsagar! |
Thanks Martin! Glad we were able to help out here 🙂 |
Reformatting with a new version of black to make CI happy. I have also updated pre-commit to the correct black repo.
Also adding some adaptations to dask main (#48)
Closes #48