Skip to content

[SPARK-56928][PYTHON] Increase minimum numpy version to 1.23.2#55959

Closed
gaogaotiantian wants to merge 3 commits into
apache:masterfrom
gaogaotiantian:upgrade-min-numpy-ver
Closed

[SPARK-56928][PYTHON] Increase minimum numpy version to 1.23.2#55959
gaogaotiantian wants to merge 3 commits into
apache:masterfrom
gaogaotiantian:upgrade-min-numpy-ver

Conversation

@gaogaotiantian
Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Increase minimum version of numpy to 1.23.2.

Why are the changes needed?

1.23.2 is the first numpy version that supports 3.11. We now remove the support for 3.10, it doesn't make sense to keep numpy version that low. This is actually breaking our CI - https://github.com/apache/spark/actions/runs/26047920278/job/76576848283

Does this PR introduce any user-facing change?

No.

How was this patch tested?

CI.

Was this patch authored or co-authored using generative AI tooling?

No.

@dongjoon-hyun
Copy link
Copy Markdown
Member

Thank you, @gaogaotiantian .

Copy link
Copy Markdown
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update here too.

`numpy` >=1.22 Required for MLlib DataFrame-based API

@dongjoon-hyun
Copy link
Copy Markdown
Member

cc @peter-toth

@gaogaotiantian
Copy link
Copy Markdown
Contributor Author

Updated the rst file and a few github actions.

dongjoon-hyun pushed a commit that referenced this pull request May 19, 2026
### What changes were proposed in this pull request?

Increase minimum version of `numpy` to 1.23.2.

### Why are the changes needed?

1.23.2 is the first numpy version that supports 3.11. We now remove the support for 3.10, it doesn't make sense to keep numpy version that low. This is actually breaking our CI - https://github.com/apache/spark/actions/runs/26047920278/job/76576848283

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

CI.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #55959 from gaogaotiantian/upgrade-min-numpy-ver.

Authored-by: Tian Gao <gaogaotiantian@hotmail.com>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit 51ff9a1)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
@dongjoon-hyun
Copy link
Copy Markdown
Member

Merged to master/4.x.

@peter-toth
Copy link
Copy Markdown
Contributor

Late LGTM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants