Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: values are dropped when grouping on a variables with NaNs #38227

Closed
randomgambit opened this issue Dec 2, 2020 · 3 comments · Fixed by #38257
Closed

BUG: values are dropped when grouping on a variables with NaNs #38227

randomgambit opened this issue Dec 2, 2020 · 3 comments · Fixed by #38257
Labels
Apply Apply, Aggregate, Transform Bug Groupby
Milestone

Comments

@randomgambit
Copy link

Hello there,

Consider this simple example

pd.__version__
Out[158]: '1.1.3'

df = pd.DataFrame({'col': [1,2,3],
                   'group' : ['a',np.NaN,'b']})

df
Out[160]: 
   col group
0    1     a
1    2   NaN
2    3     b

df.groupby('group').apply(lambda x: x)
Out[161]: 
   col group
0  1.0     a
1  NaN   NaN
2  3.0     b

There are a few things that are puzzling.

  1. I thought groupby would drop the NA groups by default
  2. Why is the variable col set to missing?

This looks like a bug to me, unless I am missing something (apologies if this is the case)
What do you think?

Thanks!

@randomgambit randomgambit added Bug Needs Triage Issue that has not been reviewed by a pandas team member labels Dec 2, 2020
@jreback
Copy link
Contributor

jreback commented Dec 2, 2020

pls try this in master

@randomgambit
Copy link
Author

1.1.4 ?

@phofl
Copy link
Member

phofl commented Dec 2, 2020

Persists on master

@jorisvandenbossche jorisvandenbossche added Groupby and removed Needs Triage Issue that has not been reviewed by a pandas team member labels Dec 2, 2020
@rhshadrach rhshadrach added the Apply Apply, Aggregate, Transform label Dec 3, 2020
@jreback jreback added this to the 1.2 milestone Dec 4, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Apply Apply, Aggregate, Transform Bug Groupby
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants