New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-38036][SQL][TESTS] Refactor VersionsSuite
to HiveClientSuite
and make it a subclass of HiveVersionSuite
#35335
Conversation
VersionsSuite
to HiveClientSuite
and make it a subclass of HiveVersionSuite
VersionsSuite
to HiveClientSuite
and make it a subclass of HiveVersionSuite
private var versionSpark: TestHiveVersion = null | ||
|
||
versions.foreach { version => | ||
test(s"$version: create client") { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The test cases in versions.foreach
have been moved to HiveClientSuite.scala
// Since Hive 3.0, HIVE-19310 skipped `ensureDbInit` if `hive.in.test=false`. | ||
hadoopConf.set("hive.in.test", "true") | ||
// Since HIVE-17626(Hive 3.0.0), need to set hive.query.reexecution.enabled=false. | ||
hadoopConf.set("hive.query.reexecution.enabled", "false") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hadoopConf.set("hive.query.reexecution.enabled", "false")
move to HiveVersionSuite.scala
as default configuration.
} | ||
} | ||
|
||
test("success sanity check") { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
success sanity check
, hadoop configuration preserved
, override useless and side-effect hive configurations
and failure sanity check
move to HiveClientSuites
} | ||
} | ||
|
||
test("create client") { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The $version:
prefix is removed because the subclass of HiveVersionSuite
will print this prefix by default, other cases in this file are similar
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
thanks for finishing this TODO!
merging to master! |
thanks @cloud-fan |
### What changes were proposed in this pull request? This PR aims to clean up the removed `VersionsSuite` reference. ### Why are the changes needed? At Apache Spark 3.3.0, `VersionsSuite` is removed via SPARK-38036 . - #35335 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #45800 from dongjoon-hyun/SPARK-47676. Authored-by: Dongjoon Hyun <dhyun@apple.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
### What changes were proposed in this pull request? This PR aims to clean up the removed `VersionsSuite` reference. ### Why are the changes needed? At Apache Spark 3.3.0, `VersionsSuite` is removed via SPARK-38036 . - #35335 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #45800 from dongjoon-hyun/SPARK-47676. Authored-by: Dongjoon Hyun <dhyun@apple.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com> (cherry picked from commit 128f74b) Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
### What changes were proposed in this pull request? This PR aims to clean up the removed `VersionsSuite` reference. ### Why are the changes needed? At Apache Spark 3.3.0, `VersionsSuite` is removed via SPARK-38036 . - #35335 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass the CIs. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #45800 from dongjoon-hyun/SPARK-47676. Authored-by: Dongjoon Hyun <dhyun@apple.com> Signed-off-by: Dongjoon Hyun <dhyun@apple.com> (cherry picked from commit 128f74b) Signed-off-by: Dongjoon Hyun <dhyun@apple.com>
What changes were proposed in this pull request?
There is a TODO in
VersionsSuite
:HiveClientSuite
and make it a subclass ofHiveVersionSuite
this pr completed this TODO, the main change as follows:
versions.foreach
scope ofVersionsSuite
toHiveClientSuite
nestedSuites
function inHiveClientSuites
to use each hive version to test the cases inHiveClientSuite
similar asHiveClientUserNameSuites
andHivePartitionFilteringSuites
HiveClientSuites
Why are the changes needed?
Make
VersionsSuite
as a subclass ofHiveVersionSuite
to unify the test mode of multi version hiveDoes this PR introduce any user-facing change?
No.
How was this patch tested?
Before
After
The number of test cases is the same, and Suites changed from 2 to 14