[SPARK-35392][ML][PYTHON] Fix flaky tests in ml/clustering.py and ml/feature.py

[zhengruifeng]

What changes were proposed in this pull request?

This PR removes the check of summary.logLikelihood in ml/clustering.py - this GMM test is quite flaky. It fails easily e.g., if:

• change number of partitions;
• just change the way to compute the sum of weights;
• change the underlying BLAS impl

Also uses more permissive precision on Word2Vec test case.

Why are the changes needed?

To recover the build and tests.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Existing test cases.

[zhengruifeng]

ping @HyukjinKwon @srowen @viirya

[HyukjinKwon]

LGTM if tests pass

[SparkQA]

Test build #138498 has finished for PR 32533 at commit 0081246.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43018/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43018/

[HyukjinKwon]

@zhengruifeng would you mind fixing:

**********************************************************************
File "/__w/spark/spark/python/pyspark/ml/feature.py", line 4681, in __main__.Word2Vec
Failed example:
    model.getVectors().show()
Expected:
    +----+--------------------+
    |word|              vector|
    +----+--------------------+
    |   a|[0.09511678665876...|
    |   b|[-1.2028766870498...|
    |   c|[0.30153277516365...|
    +----+--------------------+
    ...
Got:
    +----+--------------------+
    |word|              vector|
    +----+--------------------+
    |   a|[0.09511695802211...|
    |   b|[-1.2028766870498...|
    |   c|[0.30153274536132...|
    +----+--------------------+
    <BLANKLINE>
**********************************************************************

too? feel free to change the JIRA.

I think we can just fix it like:

    +----+--------------------+
    |word|              vector|
    +----+--------------------+
    |   a|[0.0951 ...
    |   b|[-1.202 ...
    |   c|[0.3015 ...
    +----+--------------------+

[HyukjinKwon]

LGTM

[SparkQA]

Test build #138510 has finished for PR 32533 at commit a1fd16f.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43030/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43030/

[HyukjinKwon]

Merged to master.

[HyukjinKwon]

Thanks @zhengruifeng for fixing this!

[dongjoon-hyun]

Yes, one stone for two birds! Nice!