Skip to content

Commit

Permalink
[SPARK-47993][PYTHON][FOLLOW-UP] Update migration guide about Python …
Browse files Browse the repository at this point in the history
…3.8 dropped

### What changes were proposed in this pull request?

This PR is a followup of #46228 that updates migration guide about Python 3.8 being dropped.

### Why are the changes needed?

To guide end users about the migration to Spark 4.0.

### Does this PR introduce _any_ user-facing change?

Yes, it fixes the documentation.

### How was this patch tested?

CI in this PR.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #46545 from HyukjinKwon/SPARK-47993-followup.

Authored-by: Hyukjin Kwon <gurwls223@apache.org>
Signed-off-by: yangjie01 <yangjie01@baidu.com>
  • Loading branch information
HyukjinKwon authored and LuciferYang committed May 13, 2024
1 parent cae2248 commit acc3753
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions python/docs/source/migration_guide/pyspark_upgrade.rst
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ Upgrading PySpark
Upgrading from PySpark 3.5 to 4.0
---------------------------------

* In Spark 4.0, Python 3.8 support was dropped in PySpark.
* In Spark 4.0, the minimum supported version for Pandas has been raised from 1.0.5 to 2.0.0 in PySpark.
* In Spark 4.0, the minimum supported version for Numpy has been raised from 1.15 to 1.21 in PySpark.
* In Spark 4.0, the minimum supported version for PyArrow has been raised from 4.0.0 to 10.0.0 in PySpark.
Expand Down

0 comments on commit acc3753

Please sign in to comment.