[SPARK-32882][K8S] Remove python2 installation in K8s python image

[williamhyun]

What changes were proposed in this pull request?

This PR aims to remove python2 installation in K8s python image because spark 3.1 does not support python2.

Why are the changes needed?

This will save disk space.

BEFORE

kubespark/spark-py ... 917MB

AFTER

kubespark/spark-py ... 823MB

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Pass the Jenkins with the K8s IT.

[SparkQA]

Test build #128669 has finished for PR 29751 at commit 7b22b85.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33290/

[SparkQA]

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33292/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33290/

[SparkQA]

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/33292/

[dongjoon-hyun]

+1, LGTM.

All python related test passed and the only failed decommission test case is broken on master branch like #27735 (comment) .

KubernetesSuite:
- Run SparkPi with no resources
- Run SparkPi with a very long application name.
- Use SparkLauncher.NO_RESOURCE
- Run SparkPi with a master URL without a scheme.
- Run SparkPi with an argument.
- Run SparkPi with custom labels, annotations, and environment variables.
- All pods have the same service account by default
- Run extraJVMOptions check on driver
- Run SparkRemoteFileTest using a remote data file
- Run SparkPi with env and mount secrets.
- Run PySpark on simple pi.py example
- Run PySpark with Python3 to test a pyfiles example
- Run PySpark with memory customization
- Run in client mode.
- Start pod creation from template
- PVs with local storage
- Launcher client dependencies
- Test basic decommissioning *** FAILED ***

Thank you, @williamhyun . Merged to master for Apache Spark 3.1.0 on December 2020.