Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support non-Pandas series in dd.Series.unique #4599

Merged
merged 3 commits into from Mar 17, 2019

Conversation

Projects
None yet
2 participants
@quasiben
Copy link
Member

commented Mar 15, 2019

  • Tests added / passed
  • Passes flake8 dask

This PR add a function which creates series like objects and uses the function when creating unique values. This arises when using cudf/dask_cudf:

array([<cudf.Series nrows=3 >], dtype=object)

pd.Series of the above will fail. Instead, we can build up an equivalent series like object native to pandas and/dask-cudf

cc @mrocklin

quasiben added some commits Mar 15, 2019

@mrocklin

This comment has been minimized.

Copy link
Member

commented Mar 15, 2019

I'm somewhat concerned about creating a new function create_series_like for this. I'd like to wait on making such a new operation until we have seen a few demonstrations of a place where this would be valuable. In the mean time, I might do something like the following:

def unique(s):
    out = s.unique()
    if not is_series_like(out):
        out = pd.Series(out, name=s.name)  # or whatever
    return out

This way we're not promising to maintain a create_series_like function in the future, and the logic is nicely located near the function, rather than requiring an indirection (which I find makes things difficult to debug in the future).

@quasiben

This comment has been minimized.

Copy link
Member Author

commented Mar 15, 2019

That's a good suggestion. I'll make those changes now

@mrocklin mrocklin changed the title create a series like object Support non-Pandas series in dd.Series.unique Mar 16, 2019

@mrocklin

This comment has been minimized.

Copy link
Member

commented Mar 16, 2019

Merging in 24 hours if there are no further comments

@mrocklin mrocklin merged commit 27b0f8f into dask:master Mar 17, 2019

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details

@quasiben quasiben deleted the quasiben:fix/create_series_like branch Mar 18, 2019

jorge-pessoa pushed a commit to jorge-pessoa/dask that referenced this pull request May 14, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.