-
Notifications
You must be signed in to change notification settings - Fork 28k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-35981][PYTHON][TEST][3.2] Use check_exact=False to loosen the check precision #33193
Conversation
… precision ### What changes were proposed in this pull request? We should use `check_exact=False` because the value check in `StatsTest.test_cov_corr_meta` is too strict. ### Why are the changes needed? In some environment, the precision could be different in pandas' `DataFrame.corr` function and the test `StatsTest.test_cov_corr_meta` fails. ``` AssertionError: DataFrame.iloc[:, 0] (column name="a") are different DataFrame.iloc[:, 0] (column name="a") values are different (14.28571 %) [index]: [a, b, c, d, e, f, g] [left]: [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0] [right]: [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 4.807406715958909e-17] ``` ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Modified tests should still pass. Closes apache#33179 from ueshin/issuse/SPARK-35981/corr. Authored-by: Takuya UESHIN <ueshin@databricks.com> Signed-off-by: Hyukjin Kwon <gurwls223@apache.org>
cc @HyukjinKwon |
Test build #140592 has finished for PR 33193 at commit
|
Kubernetes integration test starting |
Kubernetes integration test status success |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1, LGTM. Merged to branch-3.2 since Jenkins passed.
…check precision ### What changes were proposed in this pull request? This is a cherry-pick of #33179. We should use `check_exact=False` because the value check in `StatsTest.test_cov_corr_meta` is too strict. ### Why are the changes needed? In some environment, the precision could be different in pandas' `DataFrame.corr` function and the test `StatsTest.test_cov_corr_meta` fails. ``` AssertionError: DataFrame.iloc[:, 0] (column name="a") are different DataFrame.iloc[:, 0] (column name="a") values are different (14.28571 %) [index]: [a, b, c, d, e, f, g] [left]: [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0] [right]: [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 4.807406715958909e-17] ``` ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Modified tests should still pass. Closes #33193 from ueshin/issuse/SPARK-35981/3.2/corr. Authored-by: Takuya UESHIN <ueshin@databricks.com> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm2
What changes were proposed in this pull request?
This is a cherry-pick of #33179.
We should use
check_exact=False
because the value check inStatsTest.test_cov_corr_meta
is too strict.Why are the changes needed?
In some environment, the precision could be different in pandas'
DataFrame.corr
function and the testStatsTest.test_cov_corr_meta
fails.Does this PR introduce any user-facing change?
No.
How was this patch tested?
Modified tests should still pass.