TST: fix CI (black, dask master)

[martinfleis]

Reformatting with a new version of black to make CI happy. I have also updated pre-commit to the correct black repo.

Also adding some adaptations to dask main (#48)

Closes #48

[martinfleis]

I am thinking whether we should pin the version of black as we have it in geopandas or not. I am not sure about that, it can easily remain pinned for years unnoticed.

[jakirkham]

Should fix the last error on CI

[martinfleis]

@jakirkham I did catch those as well but there seems to be a bit more work apparently :D.

[jakirkham]

Ok will keep an eye on it ;)

Interestingly I can't mark my own comments as resolved (sorry about that)

[galipremsagar]

With latest dask, make_meta_obj is a dispatch to handle meta creation for arbitrary object (for ex: object, int, dict..). make_meta_obj is being used to register against geopandas.GeoSeries & geopandas.GeoDataFrame - This will lead to incorrect behavior which we are seeing now in CI. Instead it has to be something like:

from dask.dataframe.utils import make_meta, make_meta_obj

@make_meta.register((geopandas.GeoSeries, geopandas.GeoDataFrame))
def make_meta_geodataframe(df, index=None):
    return df.head(0)

@make_meta_obj.register((geopandas.GeoSeries, geopandas.GeoDataFrame, <any other objects of geopandas type>,...)):
def make_meta_object(x, index=None):
    if is_arraylike(x) and x.shape:
        return x[:0]

    if index is not None:
        index = make_meta(index)

    if isinstance(x, dict):
        return pd.DataFrame(
            {c: _empty_series(c, d, index=index) for (c, d) in x.items()}, index=index
        )
    if isinstance(x, tuple) and len(x) == 2:
        return _empty_series(x[0], x[1], index=index)
    elif isinstance(x, (list, tuple)):
        if not all(isinstance(i, tuple) and len(i) == 2 for i in x):
            raise ValueError(
                "Expected iterable of tuples of (name, dtype), got {0}".format(x)
            )
        return pd.DataFrame(
            {c: _empty_series(c, d, index=index) for (c, d) in x},
            columns=[c for c, d in x],
            index=index,
        )
    elif not hasattr(x, "dtype") and x is not None:
        # could be a string, a dtype object, or a python type. Skip `None`,
        # because it is implictly converted to `dtype('f8')`, which we don't
        # want here.
        try:
            dtype = np.dtype(x)
            return _scalar_from_dtype(dtype)
        except Exception:
            # Continue on to next check
            pass

    if is_scalar(x):
        return _nonempty_scalar(x)

    raise TypeError("Don't know how to create metadata from {0}".format(x))

[martinfleis]

Thanks @galipremsagar! I have fixed make_meta mismatch so all apart from one test work. That one returns a scalar (BaseGeometry) as a result of a reduction but while stable dask is fine with shapely.geometry object as meta dask@main complains and I haven't figured out how to adapt that yet. I'll continue tomorrow.

[martinfleis]

Thanks for the help @jakirkham & @galipremsagar!

[jakirkham]

Thanks Martin! Glad we were able to help out here 🙂