Skip to content

Fix groupby multilevel issues#1914

Merged
mrocklin merged 3 commits intodask:masterfrom
sinhrks:groupby
Feb 14, 2017
Merged

Fix groupby multilevel issues#1914
mrocklin merged 3 commits intodask:masterfrom
sinhrks:groupby

Conversation

@sinhrks
Copy link
Copy Markdown
Member

@sinhrks sinhrks commented Jan 14, 2017

Copy link
Copy Markdown
Member

@mrocklin mrocklin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally this looks good to me.

if pd.__version__ == '0.18.0':
from pandas.core import format
else:
from pandas.formats import format
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that @jcrist is considering dropping support for Pandas < 0.19

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, pandas < 0.19 support is going away. I hope to merge that in the next day or two (if all goes well). Could either keep the pandas < 0.19 support, and I'll remove it in that PR, or just wait until that's merged beforehand.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Support for pandas < 0.19 has been removed, so these version checks can go away :).

@sinhrks sinhrks force-pushed the groupby branch 2 times, most recently from f0c9fd7 to 8002271 Compare January 21, 2017 01:00
@mrocklin
Copy link
Copy Markdown
Member

What's the status here. Is this good to merge?

@mrocklin
Copy link
Copy Markdown
Member

cc @jreback , not sure if you're still interested in this problem.

Other keywords passed to groupby
"""
def __init__(self, df, index=None, slice=None, **kwargs):
def __init__(self, df, by=None, slice=None, axis=0, level=None,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a little clunky to have to name these args (since they are unsupported). Are these remaining kwargs actually used anywhere? Whey not simply list the acceptable ones? (and it will show a TypeError if you pass not named ones)

@jcrist
Copy link
Copy Markdown
Member

jcrist commented Jan 30, 2017

@sinhrks, what's the status of this? Besides removing the pandas version check, and maybe responding to Jeff's request (I'm fine with the way it is, but don't have a strong opinion), this looks good to me.

@sinhrks
Copy link
Copy Markdown
Member Author

sinhrks commented Feb 2, 2017

Let me update to meet @jreback 's request.

@mrocklin
Copy link
Copy Markdown
Member

Merging soon if no further comments. My apologies for the long delay.

@mrocklin mrocklin merged commit 2502c3e into dask:master Feb 14, 2017
@philippjfr
Copy link
Copy Markdown
Contributor

@sinhrks Quick question regarding these changes. I noticed that as part of this PR you are now dropping all kwargs which were previously passed down to the groupby, including sort. I'm wondering whether this was on purpose or merely an oversight. In version 0.12.0 the argument seemed to be completely ignored but in 0.13.0 it appeared to be working according to my unit tests, which started matching those I have for pandas. My question is whether sort was somehow supported by accident or was only partially working, and was therefore removed on purpose or whether it can actually be supported and was dropped by accident.

@mrocklin
Copy link
Copy Markdown
Member

mrocklin commented Mar 2, 2017

This seems like it would work if there was a single output partition?

@mrocklin
Copy link
Copy Markdown
Member

mrocklin commented Mar 2, 2017

But I suppose one could always compute and then sort as well

@sinhrks sinhrks added this to the 0.14.0 milestone Mar 4, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants