-
Notifications
You must be signed in to change notification settings - Fork 28.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-47305][SQL][TESTS][FOLLOWUP][3.4] Fix the compilation error related to PropagateEmptyRelationSuite
#45428
Conversation
PropagateEmptyRelationSuite
PropagateEmptyRelationSuite
PropagateEmptyRelationSuite
PropagateEmptyRelationSuite
This is my first time handling such a situation, is it better to create a new Jira or is it better as a FOLLOWUP of SPARK-47305? Other branches do not have this issue |
also cc @dongjoon-hyun |
Thanks @HyukjinKwon |
FOLLOWUP tag should be OK. Thanks for handling this. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1 pending CI
…lated to `PropagateEmptyRelationSuite` ### What changes were proposed in this pull request? #45406 has been backported to branch-3.4, where the newly added test case in `PropagateEmptyRelationSuite` uses `DataTypeUtils`, but `DataTypeUtils` is a utility class added in Apache Spark 3.5(SPARK-44475), so this triggered a compilation failure in branch-3.4: - https://github.com/apache/spark/actions/runs/8183755511/job/22377119069 ``` [error] /home/runner/work/spark/spark/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/PropagateEmptyRelationSuite.scala:229:27: not found: value DataTypeUtils [error] val schemaForStream = DataTypeUtils.fromAttributes(outputForStream) [error] ^ [error] /home/runner/work/spark/spark/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/PropagateEmptyRelationSuite.scala:233:26: not found: value DataTypeUtils [error] val schemaForBatch = DataTypeUtils.fromAttributes(outputForBatch) [error] ^ [info] done compiling [info] compiling 1 Scala source to /home/runner/work/spark/spark/connector/connect/common/target/scala-2.12/test-classes ... [info] compiling 25 Scala sources and 1 Java source to /home/runner/work/spark/spark/connector/connect/client/jvm/target/scala-2.12/classes ... [info] done compiling [error] two errors found ``` Therefore, this PR changes to use the `StructType.fromAttributes` function to fix the compilation failure." ### Why are the changes needed? Fix the compilation failure in branch-3.4 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Pass Github Actions ### Was this patch authored or co-authored using generative AI tooling? No Closes #45428 from LuciferYang/SPARK-47305-FOLLOWUP-34. Authored-by: yangjie01 <yangjie01@baidu.com> Signed-off-by: yangjie01 <yangjie01@baidu.com>
Merged into branch-3.4 for fix compilation failure. Thanks @HyukjinKwon @yaooqinn @HeartSaVioR @zhengruifeng |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1, LGTM. Thanks!
…lated to `PropagateEmptyRelationSuite` ### What changes were proposed in this pull request? apache#45406 has been backported to branch-3.4, where the newly added test case in `PropagateEmptyRelationSuite` uses `DataTypeUtils`, but `DataTypeUtils` is a utility class added in Apache Spark 3.5(SPARK-44475), so this triggered a compilation failure in branch-3.4: - https://github.com/apache/spark/actions/runs/8183755511/job/22377119069 ``` [error] /home/runner/work/spark/spark/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/PropagateEmptyRelationSuite.scala:229:27: not found: value DataTypeUtils [error] val schemaForStream = DataTypeUtils.fromAttributes(outputForStream) [error] ^ [error] /home/runner/work/spark/spark/sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/PropagateEmptyRelationSuite.scala:233:26: not found: value DataTypeUtils [error] val schemaForBatch = DataTypeUtils.fromAttributes(outputForBatch) [error] ^ [info] done compiling [info] compiling 1 Scala source to /home/runner/work/spark/spark/connector/connect/common/target/scala-2.12/test-classes ... [info] compiling 25 Scala sources and 1 Java source to /home/runner/work/spark/spark/connector/connect/client/jvm/target/scala-2.12/classes ... [info] done compiling [error] two errors found ``` Therefore, this PR changes to use the `StructType.fromAttributes` function to fix the compilation failure." ### Why are the changes needed? Fix the compilation failure in branch-3.4 ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? Pass Github Actions ### Was this patch authored or co-authored using generative AI tooling? No Closes apache#45428 from LuciferYang/SPARK-47305-FOLLOWUP-34. Authored-by: yangjie01 <yangjie01@baidu.com> Signed-off-by: yangjie01 <yangjie01@baidu.com>
What changes were proposed in this pull request?
#45406 has been backported to branch-3.4, where the newly added test case in
PropagateEmptyRelationSuite
usesDataTypeUtils
, butDataTypeUtils
is a utility class added in Apache Spark 3.5(SPARK-44475), so this triggered a compilation failure in branch-3.4:Therefore, this PR changes to use the
StructType.fromAttributes
function to fix the compilation failure."Why are the changes needed?
Fix the compilation failure in branch-3.4
Does this PR introduce any user-facing change?
No
How was this patch tested?
Pass Github Actions
Was this patch authored or co-authored using generative AI tooling?
No