Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-27101][PYTHON] Drop the created database after the test in test_session #24021

Closed
wants to merge 1 commit into from
Closed

[SPARK-27101][PYTHON] Drop the created database after the test in test_session #24021

wants to merge 1 commit into from

Conversation

sandeep-katta
Copy link
Contributor

What changes were proposed in this pull request?

Cleaning the testcase, drop the database after use

How was this patch tested?

existing UT

@sandeep-katta
Copy link
Contributor Author

cc @HyukjinKwon @ueshin

@sandeep-katta
Copy link
Contributor Author

this is required for 22466. Python testcases are failing

HyukjinKwon
HyukjinKwon reviewed Mar 8, 2019
View reviewed changes
python/pyspark/sql/tests/test_session.py Outdated
self.assertEqual(spark.range(3).count(), 3)
finally:
spark.stop()
with self.database("test_db"):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it because the database wasn't cleaned? In that case, you can just add class SparkSessionTests3(unittest.TestCase, SQLTestUtils):.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SQLTestUtils excepts spark session to be present Code here. So have ReusedSQLTestCase which will create spark session and inherits unittest.TestCase, SQLTestUtils

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This set of tests are specific to Spark sessions. Let's don't make another Spark session before the tests. You can just manually drop the database explicitly in finally

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Handled

@HyukjinKwon
Copy link
Member

ok to test

@SparkQA
Copy link

SparkQA commented Mar 8, 2019

Test build #103205 has finished for PR 24021 at commit 4dbda53.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
  • class SparkSessionTests3(ReusedSQLTestCase):

@sandeep-katta
Copy link
Contributor Author

retest this please

@SparkQA
Copy link

SparkQA commented Mar 8, 2019

Test build #103208 has finished for PR 24021 at commit d9fcaec.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon HyukjinKwon changed the title [SPARK-27101][Pyspark]Cleaning up the python testcases [SPARK-27101][PYTHON] Drop the created database after the test in test_session Mar 9, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@HyukjinKwon HyukjinKwon HyukjinKwon approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
3 participants
@sandeep-katta @HyukjinKwon @SparkQA