Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Break isnull type operations out into a separate expression #697

Closed
cpcloud opened this issue Sep 30, 2014 · 6 comments · Fixed by #733
Closed

Break isnull type operations out into a separate expression #697

cpcloud opened this issue Sep 30, 2014 · 6 comments · Fixed by #733
Assignees
Milestone

Comments

@cpcloud
Copy link
Member

cpcloud commented Sep 30, 2014

Currently the isnan method uses np.isnan or math.isnan. We should add an additional isnull that checks for null values.

Most backends have something that corresponds to this operation.

cc @quasiben @chdoig @mrocklin

@cpcloud cpcloud self-assigned this Sep 30, 2014
@cpcloud cpcloud added this to the Release 0.6.4 milestone Sep 30, 2014
@mrocklin
Copy link
Member

If this is commonly used in the following way

col[~col.isnan]

Then we might consider the following operation instead

col.nonnull

@cpcloud
Copy link
Member Author

cpcloud commented Sep 30, 2014

pandas actually has two names for this dropna and valid

just fyi

@cpcloud
Copy link
Member Author

cpcloud commented Sep 30, 2014

aesthetically not a fan of nonnull, but i could go for dropnull

@mrocklin
Copy link
Member

Yeah, dropnull sounds better. We should shop around a few names to other folks at some point.

What is more convenient from a computational perspective, isnull or dropnull? This should probably drive the creation of the Expression. The syntax/names can be decided after the fact.

@cpcloud
Copy link
Member Author

cpcloud commented Sep 30, 2014

I think both should exist. Isnull gives you an indicator variable and drop null gives you values where the result of isnull is false.

@mrocklin
Copy link
Member

In regards to syntax I agree. In regards to computation I think that this requires us to look at how computational backends handle things. For example. If everything is like numpy then we don't need to bother with dropnull; isnull is sufficient.

@cpcloud cpcloud modified the milestones: Release 0.6.4, Release 0.7 Oct 7, 2014
@cpcloud cpcloud modified the milestones: Release 0.7, 0.7.1, Release 0.8 Dec 21, 2014
@cpcloud cpcloud modified the milestones: Release 0.8, Release 0.9 Mar 13, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants