-
Notifications
You must be signed in to change notification settings - Fork 28k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-46607][PYTHON][TESTS] Check the testing mode #44611
Conversation
7a1df12
to
28ca22b
Compare
28ca22b
to
7320739
Compare
@@ -19,10 +19,14 @@ | |||
|
|||
from pyspark.pandas.tests.groupby.test_grouping import GroupingTestsMixin | |||
from pyspark.testing.connectutils import ReusedConnectTestCase | |||
from pyspark.testing.pandasutils import PandasOnSparkTestCase | |||
from pyspark.testing.pandasutils import PandasOnSparkTestUtils |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
manually check that, before this fix this test suite fails with
test_getitem (pyspark.pandas.tests.connect.groupby.test_parity_grouping.GroupingParityTests.test_getitem) ... ok (0.022s)
test_assert_vanilla_mode (pyspark.testing.pandasutils.PandasOnSparkTestCase.test_assert_vanilla_mode) ... ok (0.000s)
test_assert_remote_mode (pyspark.testing.connectutils.ReusedConnectTestCase.test_assert_remote_mode) ... ok (0.000s)
======================================================================
FAIL [0.003s]: test_assert_remote_mode (pyspark.pandas.tests.connect.groupby.test_parity_grouping.GroupingParityTests.test_assert_remote_mode)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/ruifeng.zheng/Dev/spark/python/pyspark/testing/connectutils.py", line 199, in test_assert_remote_mode
self.assertTrue(is_remote())
AssertionError: False is not true
----------------------------------------------------------------------
Ran 6 tests in 8.455s
cc @HyukjinKwon and @itholic please take a look |
def test_assert_remote_mode(self): | ||
from pyspark.sql import is_remote | ||
|
||
self.assertTrue(is_remote()) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
Merged to master. |
thanks @HyukjinKwon and @itholic for reivews |
What changes were proposed in this pull request?
1, check the testing mode:
test_assert_vanilla_mode
inReusedPySparkTestCase
test_assert_remote_mode
inReusedConnectTestCase
use different function names in case a test suite inherit them both
2, fix the incorrect testing mode introduced in #44196
Why are the changes needed?
incorrect usage of
PandasOnSparkTestCase
(subclass ofReusedPySparkTestCase
) in parity tests cause test with vanilla Spark Session:#44196
#44592
Does this PR introduce any user-facing change?
no
How was this patch tested?
ci, added UT
Was this patch authored or co-authored using generative AI tooling?
no