New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Groupby filter changes ordering #4621

Closed
hayd opened this Issue Aug 21, 2013 · 9 comments

Comments

Projects
None yet
4 participants
@hayd
Contributor

hayd commented Aug 21, 2013

At the moment filter reorders wrt the groups.

Example:

In [1]: data = pd.DataFrame(
    {'pid' : [1,1,1,2,2,3,3,3],
     'tag' : [23,45,62,24,45,34,25,62],
     })

In [2]: g = data.groupby('tag')

In [3]: g.filter(lambda x: len(x) > 1)
Out[3]: 
   pid  tag
1    1   45
4    2   45
2    1   62
7    3   62

If there is a way to efficiently keep the order that would be ideal I think, failing that sort back afterwards (but being wary of sorting with dupe/unordered indexes).

cc #3680

@jtratner

This comment has been minimized.

Contributor

jtratner commented Aug 21, 2013

@hayd What should it output instead?

@hayd

This comment has been minimized.

Contributor

hayd commented Aug 22, 2013

Basically:

In [12]: g.filter(lambda x: len(x) > 1, dropna=False).dropna()
Out[12]:
   pid  tag
1    1   45
2    1   62
4    2   45
7    3   62

@jreback suggested in #3680 could be more efficient way...

@jreback

This comment has been minimized.

Contributor

jreback commented Sep 27, 2013

cc @danielballan can you have a look?

@danielballan

This comment has been minimized.

Contributor

danielballan commented Sep 27, 2013

On it.

@jreback

This comment has been minimized.

Contributor

jreback commented Oct 2, 2013

@jreback

This comment has been minimized.

Contributor

jreback commented Oct 7, 2013

hows this coming?

@danielballan

This comment has been minimized.

Contributor

danielballan commented Oct 13, 2013

I will open this up once #5096 is merged....

@jreback

This comment has been minimized.

Contributor

jreback commented Oct 13, 2013

@danielballan go for it!

@danielballan

This comment has been minimized.

Contributor

danielballan commented Oct 14, 2013

@hayd, please take a look at #5222 and see if my tests look good to you.

@jreback jreback closed this in 62b7816 Oct 15, 2013

jreback added a commit that referenced this issue Oct 15, 2013

Merge pull request #5222 from danielballan/filter-maintains-ordering
BUG: Groupby filter maintains ordering, closes #4621
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment