[SPARK-51532][INFRA] Restore the installation of Python 3.11 in macOS daily test#50293
Closed
LuciferYang wants to merge 2 commits intoapache:masterfrom
Closed
[SPARK-51532][INFRA] Restore the installation of Python 3.11 in macOS daily test#50293LuciferYang wants to merge 2 commits intoapache:masterfrom
LuciferYang wants to merge 2 commits intoapache:masterfrom
Conversation
Contributor
Author
dongjoon-hyun
approved these changes
Mar 17, 2025
Member
dongjoon-hyun
left a comment
There was a problem hiding this comment.
+1, LGTM. Thank you, @LuciferYang .
Member
|
Merged to master. |
Contributor
Author
|
Thank you @dongjoon-hyun |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
This pr enables macOS to use
actions/setup-pythonto install Python 3.11, so that the relevant modules can be tested using Python 3.11.Why are the changes needed?
Although the macOS daily test uses macOS-15, which comes with Python 3.9 to Python 3.13 installed by default, the
python3command points to Python 3.13 by default instead of Python 3.11, which is required for the tests. This leads to some unexpected behaviors during testing. For example, I encountered the following logs while testing:This is because the relevant modules were installed using Python 3.11, but the
python3is actually Python 3.13. Therefore, this pull request restores the installation process for Python 3.11 on macOS, so that the tests also use Python 3.11.Does this PR introduce any user-facing change?
No, just for test
How was this patch tested?
Scalar Pandas UDF is skipped because pyspark,pandas and/or pyarrow were not available in [python3]. !!! IGNORED !!!Was this patch authored or co-authored using generative AI tooling?
No