Skip to content

Conversation

LuciferYang
Copy link
Contributor

@LuciferYang LuciferYang commented Sep 23, 2025

What changes were proposed in this pull request?

This pr aim to fix compilation errors in SparkPipelinesSuite

Why are the changes needed?

Restore GA:

[error] /home/runner/work/spark/spark/core/src/test/scala/org/apache/spark/deploy/SparkPipelinesSuite.scala:123:63: unknown parameter name: sparkHome
[error]       SparkPipelines.constructSparkSubmitArgs(args, sparkHome = "abc")
[error]                                                               ^
[error] /home/runner/work/spark/spark/core/src/test/scala/org/apache/spark/deploy/SparkPipelinesSuite.scala:127:63: unknown parameter name: sparkHome
[error]       SparkPipelines.constructSparkSubmitArgs(args, sparkHome = "abc")
[error]                                                               ^
[error] /home/runner/work/spark/spark/core/src/test/scala/org/apache/spark/deploy/SparkPipelinesSuite.scala:131:63: unknown parameter name: sparkHome
[error]       SparkPipelines.constructSparkSubmitArgs(args, sparkHome = "abc") ==
[error]                                                               ^
[error] /home/runner/work/spark/spark/core/src/test/scala/org/apache/spark/deploy/SparkPipelinesSuite.scala:142:63: unknown parameter name: sparkHome
[error]       SparkPipelines.constructSparkSubmitArgs(args, sparkHome = "abc") ==
[error]                                                               ^
[error] /home/runner/work/spark/spark/core/src/test/scala/org/apache/spark/deploy/SparkPipelinesSuite.scala:153:63: unknown parameter name: sparkHome
[error]       SparkPipelines.constructSparkSubmitArgs(args, sparkHome = "abc") ==
[error]                                                               ^
[error] /home/runner/work/spark/spark/core/src/test/scala/org/apache/spark/deploy/SparkPipelinesSuite.scala:164:63: unknown parameter name: sparkHome
[error]       SparkPipelines.constructSparkSubmitArgs(args, sparkHome = "abc") ==
[error]                                                               ^
[error] /home/runner/work/spark/spark/core/src/test/scala/org/apache/spark/deploy/SparkPipelinesSuite.scala:175:63: unknown parameter name: sparkHome
[error]       SparkPipelines.constructSparkSubmitArgs(args, sparkHome = "abc") ==
[error]                                                               ^

Does this PR introduce any user-facing change?

No

How was this patch tested?

  • Pass GitHub Actions
  • Manual checked build/sbt clean "core/testOnly org.apache.spark.deploy.SparkPipelinesSuite"
WARNING: Using incubator modules: jdk.incubator.vector
[info] SparkPipelinesSuite:
[info] - only spark submit args (20 milliseconds)
[info] - only pipelines args (1 millisecond)
[info] - spark-submit and pipelines args (0 milliseconds)
[info] - class arg prohibited (2 milliseconds)
[info] - spark.api.mode arg (1 millisecond)
[info] - name arg (0 milliseconds)
[info] Run completed in 692 milliseconds.
[info] Total number of tests run: 6
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 6, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.

Was this patch authored or co-authored using generative AI tooling?

No

@github-actions github-actions bot added the CORE label Sep 23, 2025
@LuciferYang
Copy link
Contributor Author

Thank you @sarutak ~
also cc @sryza FYI

@LuciferYang
Copy link
Contributor Author

I will merge this PR to fix the Github Actions once the tests in SparkPipelinesSuite pass.

@LuciferYang LuciferYang changed the title [SPARK-53498][SDP][TESTS][FOLLOWUP] Fix compilation errors in SparkPipelinesSuite [SPARK-53498][CORE][TESTS][FOLLOWUP] Fix compilation errors in SparkPipelinesSuite Sep 23, 2025
Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM. Thank you for the quick fix, @LuciferYang .

Copy link
Contributor

@sryza sryza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 thanks for fixing 🙏

@LuciferYang
Copy link
Contributor Author

image

SparkPipelinesSuite passed

@LuciferYang LuciferYang changed the title [SPARK-53498][CORE][TESTS][FOLLOWUP] Fix compilation errors in SparkPipelinesSuite [SPARK-53516][CORE][TESTS][FOLLOWUP] Fix compilation errors in SparkPipelinesSuite Sep 23, 2025
@LuciferYang
Copy link
Contributor Author

Merged into master. Thanks @sarutak @dongjoon-hyun @sryza

@zhengruifeng
Copy link
Contributor

late LGTM, thanks @LuciferYang !

@LuciferYang
Copy link
Contributor Author

Thank you @zhengruifeng

@LuciferYang LuciferYang deleted the SPARK-53498-FOLLOWUP branch September 24, 2025 11:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants