Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-22700][ML] Bucketizer.transform incorrectly drops row containing NaN #19894

Closed
wants to merge 2 commits into from

Conversation

@zhengruifeng
Copy link
Contributor

zhengruifeng commented Dec 5, 2017

What changes were proposed in this pull request?

only drops the rows containing NaN in the input columns

How was this patch tested?

existing tests and added tests

@SparkQA

This comment has been minimized.

Copy link

SparkQA commented Dec 5, 2017

Test build #84481 has finished for PR 19894 at commit 01604c7.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.
@SparkQA

This comment has been minimized.

Copy link

SparkQA commented Dec 5, 2017

Test build #84485 has finished for PR 19894 at commit eaebedb.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.
@zhengruifeng

This comment has been minimized.

Copy link
Contributor Author

zhengruifeng commented Dec 6, 2017

ping @MLnick ?

@hhbyyh

This comment has been minimized.

Copy link
Contributor

hhbyyh commented Dec 8, 2017

LGTM. Good fix.

Copy link
Contributor

WeichenXu123 left a comment

LGTM.

@MLnick

This comment has been minimized.

Copy link
Contributor

MLnick commented Dec 13, 2017

LGTM thanks! Merged to master

@asfgit asfgit closed this in 8743509 Dec 13, 2017
@zhengruifeng zhengruifeng deleted the zhengruifeng:bucketizer_nan branch Dec 13, 2017
@jkbradley

This comment has been minimized.

Copy link
Member

jkbradley commented Feb 8, 2018

I'm going to backport this to 2.2 since it's a correctness bug.
Update: Actually, it doesn't backport cleanly. I can do a backport PR in a few days, or feel free to go ahead and send one. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.