[SPARK-50019][INFRA] Install BASIC_PIP_PKGS except pyarrow in Python 3.13 image

[dongjoon-hyun]

What changes were proposed in this pull request?

This PR aims to install BASIC_PIP_PKGS except pyarrow in Python 3.13 image.

Why are the changes needed?

• https://github.com/apache/spark/actions/runs/11392144577/job/31698382766

Traceback (most recent call last):
  File "/__w/spark/spark/python/pyspark/sql/pandas/utils.py", line 28, in require_minimum_pandas_version
    import pandas
ModuleNotFoundError: No module named 'pandas'

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Manual check with the built image of this PR builder.

$ docker run -it --rm ghcr.io/dongjoon-hyun/apache-spark-ci-image:master-11392974455 python3.13 -m pip list | grep pandas
pandas                   2.2.3

Was this patch authored or co-authored using generative AI tooling?

No.

[dongjoon-hyun]

Could you review this PR, @viirya ?

[HyukjinKwon]

Merged to master.

[dongjoon-hyun]

Thank you, @HyukjinKwon !

[viirya]

Looks good to me.

[dongjoon-hyun]

Thank you, @viirya !

[dongjoon-hyun]

For the record, Today's pyspark-pandas test pipeline passed with this PR.

• https://github.com/apache/spark/actions/runs/11410197427/job/31752135841