Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-22332][ML][TEST] Fix NaiveBayes unit test occasionly fail (cause by test dataset not deterministic) #19558

Closed
wants to merge 3 commits into from

Conversation

WeichenXu123
Copy link
Contributor

@WeichenXu123 WeichenXu123 commented Oct 23, 2017

What changes were proposed in this pull request?

Fix NaiveBayes unit test occasionly fail:
Set seed for BrzMultinomial.sample, make generateNaiveBayesInput output deterministic dataset.
(If we do not set seed, the generated dataset will be random, and the model will be possible to exceed the tolerance in the test, which trigger this failure)

How was this patch tested?

Manually run tests multiple times and check each time output models contains the same values.

@SparkQA
Copy link

SparkQA commented Oct 23, 2017

Test build #82974 has finished for PR 19558 at commit de780e3.

  • This patch fails to build.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Oct 23, 2017

Test build #82978 has finished for PR 19558 at commit be2606b.

  • This patch fails to build.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Oct 23, 2017

Test build #82983 has finished for PR 19558 at commit 331d026.

  • This patch fails Scala style tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Oct 23, 2017

Test build #82985 has finished for PR 19558 at commit 3ea8c50.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@WeichenXu123
Copy link
Contributor Author

cc @jkbradley @MrBago

@jkbradley
Copy link
Member

LGTM
Tested locally, and it fixed the non-determinism.
Merging with master and branch-2.2
Thanks @WeichenXu123 !

asfgit pushed a commit that referenced this pull request Oct 25, 2017
…se by test dataset not deterministic)

## What changes were proposed in this pull request?

Fix NaiveBayes unit test occasionly fail:
Set seed for `BrzMultinomial.sample`, make `generateNaiveBayesInput` output deterministic dataset.
(If we do not set seed, the generated dataset will be random, and the model will be possible to exceed the tolerance in the test, which trigger this failure)

## How was this patch tested?

Manually run tests multiple times and check each time output models contains the same values.

Author: WeichenXu <weichen.xu@databricks.com>

Closes #19558 from WeichenXu123/fix_nb_test_seed.

(cherry picked from commit 841f1d7)
Signed-off-by: Joseph K. Bradley <joseph@databricks.com>
@asfgit asfgit closed this in 841f1d7 Oct 25, 2017
MatthewRBruce pushed a commit to Shopify/spark that referenced this pull request Jul 31, 2018
…se by test dataset not deterministic)

## What changes were proposed in this pull request?

Fix NaiveBayes unit test occasionly fail:
Set seed for `BrzMultinomial.sample`, make `generateNaiveBayesInput` output deterministic dataset.
(If we do not set seed, the generated dataset will be random, and the model will be possible to exceed the tolerance in the test, which trigger this failure)

## How was this patch tested?

Manually run tests multiple times and check each time output models contains the same values.

Author: WeichenXu <weichen.xu@databricks.com>

Closes apache#19558 from WeichenXu123/fix_nb_test_seed.

(cherry picked from commit 841f1d7)
Signed-off-by: Joseph K. Bradley <joseph@databricks.com>
@WeichenXu123 WeichenXu123 deleted the fix_nb_test_seed branch April 24, 2019 21:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants