Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support non-Pandas series in dd.Series.unique #4599

merged 3 commits into from Mar 17, 2019


Copy link

@quasiben quasiben commented Mar 15, 2019

  • Tests added / passed
  • Passes flake8 dask

This PR add a function which creates series like objects and uses the function when creating unique values. This arises when using cudf/dask_cudf:

array([<cudf.Series nrows=3 >], dtype=object)

pd.Series of the above will fail. Instead, we can build up an equivalent series like object native to pandas and/dask-cudf

cc @mrocklin

Copy link

@mrocklin mrocklin commented Mar 15, 2019

I'm somewhat concerned about creating a new function create_series_like for this. I'd like to wait on making such a new operation until we have seen a few demonstrations of a place where this would be valuable. In the mean time, I might do something like the following:

def unique(s):
    out = s.unique()
    if not is_series_like(out):
        out = pd.Series(out,  # or whatever
    return out

This way we're not promising to maintain a create_series_like function in the future, and the logic is nicely located near the function, rather than requiring an indirection (which I find makes things difficult to debug in the future).

Copy link
Member Author

@quasiben quasiben commented Mar 15, 2019

That's a good suggestion. I'll make those changes now

@mrocklin mrocklin changed the title create a series like object Support non-Pandas series in dd.Series.unique Mar 16, 2019
Copy link

@mrocklin mrocklin commented Mar 16, 2019

Merging in 24 hours if there are no further comments

@mrocklin mrocklin merged commit 27b0f8f into dask:master Mar 17, 2019
2 checks passed
@quasiben quasiben deleted the fix/create_series_like branch Mar 18, 2019
jorge-pessoa pushed a commit to jorge-pessoa/dask that referenced this issue May 14, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants