-
Notifications
You must be signed in to change notification settings - Fork 28k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-47304][SQL][TESTS] Distribute tests from DataFrameSuite
to more specific suites
#45392
Conversation
DataFrameSuite
to more specific suitesDataFrameSuite
to more specific suites
DataFrameSuite
to more specific suitesDataFrameSuite
to more specific suites
sql/core/src/test/scala/org/apache/spark/sql/DataFrameShowSuite.scala
Outdated
Show resolved
Hide resolved
@@ -365,4 +367,57 @@ class DataFrameToSchemaSuite extends QueryTest with SharedSparkSession { | |||
errorClass = "NOT_NULL_CONSTRAINT_VIOLATION.MAP_VALUE", | |||
parameters = Map("columnPath" -> "`map`")) | |||
} | |||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test suite is for the df.to
API. I feel it's better to put these tests in DataFrameShowSuite
, as printSchema
is more related to df.show
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
moved
|
||
test("SPARK-32761: aggregating multiple distinct CONSTANT columns") { | ||
checkAnswer(sql("select count(distinct 2), count(distinct 2,3)"), Row(1, 1)) | ||
} | ||
} | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just for curiosity, what's the runtime of DataFrameAggregateSuite
? It has more than 2000 LOC...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
1 min 5 sec on my laptop.
Merging to master. Thank you, @cloud-fan for review. |
…ore specific suites ### What changes were proposed in this pull request? In the PR, I propose to move some tests from `DataFrameSuite` out, and distribute them across more specific tests suites and new one `DataFrameShowSuite`. ### Why are the changes needed? 1. Improve maintainability of `DataFrameSuite` 2. Speed up execution of the test suite. Execution time dropped to < 1min. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? By running the modified test suites. ### Was this patch authored or co-authored using generative AI tooling? No. Closes apache#45392 from MaxGekk/split-DataFrameSuite. Authored-by: Max Gekk <max.gekk@gmail.com> Signed-off-by: Max Gekk <max.gekk@gmail.com>
What changes were proposed in this pull request?
In the PR, I propose to move some tests from
DataFrameSuite
out, and distribute them across more specific tests suites and new oneDataFrameShowSuite
.Why are the changes needed?
DataFrameSuite
Does this PR introduce any user-facing change?
No.
How was this patch tested?
By running the modified test suites.
Was this patch authored or co-authored using generative AI tooling?
No.