Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Break isnull type operations out into a separate expression #697

Closed
cpcloud opened this issue Sep 30, 2014 · 6 comments

Comments

@cpcloud
Copy link
Member

commented Sep 30, 2014

Currently the isnan method uses np.isnan or math.isnan. We should add an additional isnull that checks for null values.

Most backends have something that corresponds to this operation.

cc @quasiben @chdoig @mrocklin

@cpcloud cpcloud self-assigned this Sep 30, 2014

@cpcloud cpcloud added this to the Release 0.6.4 milestone Sep 30, 2014

@mrocklin

This comment has been minimized.

Copy link
Member

commented Sep 30, 2014

If this is commonly used in the following way

col[~col.isnan]

Then we might consider the following operation instead

col.nonnull
@cpcloud

This comment has been minimized.

Copy link
Member Author

commented Sep 30, 2014

pandas actually has two names for this dropna and valid

just fyi

@cpcloud

This comment has been minimized.

Copy link
Member Author

commented Sep 30, 2014

aesthetically not a fan of nonnull, but i could go for dropnull

@mrocklin

This comment has been minimized.

Copy link
Member

commented Sep 30, 2014

Yeah, dropnull sounds better. We should shop around a few names to other folks at some point.

What is more convenient from a computational perspective, isnull or dropnull? This should probably drive the creation of the Expression. The syntax/names can be decided after the fact.

@cpcloud

This comment has been minimized.

Copy link
Member Author

commented Sep 30, 2014

I think both should exist. Isnull gives you an indicator variable and drop null gives you values where the result of isnull is false.

@mrocklin

This comment has been minimized.

Copy link
Member

commented Sep 30, 2014

In regards to syntax I agree. In regards to computation I think that this requires us to look at how computational backends handle things. For example. If everything is like numpy then we don't need to bother with dropnull; isnull is sufficient.

@cpcloud cpcloud modified the milestones: Release 0.6.4, Release 0.7 Oct 7, 2014

@cpcloud cpcloud modified the milestones: Release 0.7, 0.7.1, Release 0.8 Dec 21, 2014

@cpcloud cpcloud modified the milestones: Release 0.8, Release 0.9 Mar 13, 2015

@cpcloud cpcloud removed the 2 - On Deck label Apr 17, 2015

@cpcloud cpcloud closed this in #733 Aug 20, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.