Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Non-expected NaN numbers behavior for .dropna function in 0.24.2 #26472

Closed
Uiuran opened this issue May 20, 2019 · 4 comments
Closed

Non-expected NaN numbers behavior for .dropna function in 0.24.2 #26472

Uiuran opened this issue May 20, 2019 · 4 comments
Labels
Needs Info Clarification about behavior needed to assess issue

Comments

@Uiuran
Copy link

Uiuran commented May 20, 2019

Unfurtunately i cant provide the data iam using (since it's enterprise privative), but it has a lot of NaN values.

I do:
df = df.dropna(axis=0,thresh=1)

Returns a DataFrame with nan values, i can confirm this looping through the DataFrame and applying np.isnan.

Then i do:
df = df.fillna(0)

The new DataFrame has no more NaN. Why is this that dropna cant see all NaNs and fillna can ?

Thankyou in advance.

#### Problem description

[this should explain **why** the current behaviour is a problem and why the expected output is a better solution.]

**Note**: We receive a lot of issues on our GitHub tracker, so it is very possible that your issue has been posted before. Please check first before submitting so that we do not have to handle and close duplicates!

**Note**: Many problems can be resolved by simply upgrading `pandas` to the latest version. Before submitting, please check if that solution works for you. If possible, you may want to check if `master` addresses this issue, but that is not necessary.

For documentation-related issues, you can check the latest versions of the docs on `master` here:

https://pandas-docs.github.io/pandas-docs-travis/

If the issue has not been resolved there, go ahead and file it in the issue tracker.

#### Expected Output

#### Output of ``pd.show_versions()``

<details>

[paste the output of ``pd.show_versions()`` here below this line]

</details>
@TomAugspurger
Copy link
Contributor

Presumably, the issue doesn't depend on the exact data. See http://matthewrocklin.com/blog/work/2018/02/28/minimal-bug-reports

@TomAugspurger TomAugspurger added the Needs Info Clarification about behavior needed to assess issue label May 20, 2019
@gustavolinari
Copy link

Probably because of the thresh parameter. Remove it and all NaN will be gone.

@Uiuran
Copy link
Author

Uiuran commented May 26, 2019 via email

@Uiuran
Copy link
Author

Uiuran commented May 28, 2019

Without thresh it does works. Strange behaviour that of this parameter though

@Uiuran Uiuran closed this as completed May 28, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Needs Info Clarification about behavior needed to assess issue
Projects
None yet
Development

No branches or pull requests

3 participants