Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

isna() doesn't seem to work in @where statement #118

Closed
mwsohn opened this issue May 29, 2017 · 2 comments
Closed

isna() doesn't seem to work in @where statement #118

mwsohn opened this issue May 29, 2017 · 2 comments

Comments

@mwsohn
Copy link

mwsohn commented May 29, 2017

In general, it would be very nice if Query.jl provides an easy functionality by which NA values are removed from a DataFrame.

@davidanthoff
Copy link
Member

You need to use isnull in queries instead of isna.

Queries is using its own type DataValue to represent missing values, so even if you start with something like a DataFrame, missing values will be represented as an instance of DataValue inside the query. I would actually prefer to use isna for DataValues, but right now I can't do that without taking a dependency on DataArrays, which I don't want to do. So until I come up with some other way, you'll have to use isnull..

In general, it would be very nice if Query.jl provides an easy functionality by which NA values are removed from a DataFrame.

I assume you are looking for an easy way to remove rows that have missing values? The next version of Query (which will only work on julia 0.6) should enable something like this

@where !any(isnull.(i))

here you apply the isnull function to all columns of the source using the dot vectorization syntax from base, and then you make sure there are no trues returned. Would that address your need? Please let me know if it doesn't, or if you think this is too clumsy etc. I'm really looking for feedback on these design issues to make sure the package ends up working for folks!

@mwsohn
Copy link
Author

mwsohn commented May 29, 2017

That would be great. isnull also works well. Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants