BIGTOP-4053. Remove Python as Spark Installation Dependency

[masatana]

Description of PR

https://issues.apache.org/jira/browse/BIGTOP-4053

As discussed in the JIRA issue, I have removed the python-releated spark tests in this PR.
Please feel free to comment on that.

How was this patch tested?

On rocky-8

[rocky@ip-172-31-4-21 noarch]$ rpm -qp spark-python-3.3.4-1.el8.noarch.rpm --requires
/bin/bash
/usr/bin/env
rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib(FileDigests) <= 4.6.0-1
rpmlib(PayloadFilesHavePrefix) <= 4.0-1
rpmlib(PayloadIsXz) <= 5.2-1
spark-core = 3.3.4-1.el8

On Ubuntu-22.04

ubuntu@ip-172-31-12-120:~$ dpkg -I ./bigtop/output/spark/spark-python_3.3.4-1_all.deb
 new Debian package, version 2.0.
 size 3328910 bytes: control archive=12048 bytes.
     350 bytes,    12 lines      control
   43376 bytes,   474 lines      md5sums
 Package: spark-python
 Source: spark
 Version: 3.3.4-1
 Architecture: all
 Maintainer: Bigtop <dev@bigtop.apache.org>
 Installed-Size: 13506
 Depends: spark-core (= 3.3.4-1)
 Section: misc
 Priority: extra
 Homepage: http://spark.apache.org/
 Description: Python client for Spark
  Includes PySpark, an interactive Python shell for Spark, and related libraries

For code changes:

• Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'BIGTOP-3638. Your PR title ...')?
• Make sure that newly added files do not have any licensing issues. When in doubt refer to https://www.apache.org/licenses/

[iwasakims]

LGTM. smoke-tests passed on CentOS 7 aarach64 and Ubuntu 22.04 x86_64 on my side. Thanks for the contribution, @masatana.

[JiaLiangC]

@iwasakims Don't we need to handle the logic PYSPARK_PYTHON=python2 in the spec code?

[masatana]

Oops, I haven't catched that code.
And should we also handle the code here ?

bigtop/bigtop-packages/src/common/spark/install_spark.sh

Line 185 in e98dba6

export PYSPARK_PYTHON=python

[JiaLiangC]

Oops, I haven't catched that code. And should we also handle the code here ?

bigtop/bigtop-packages/src/common/spark/install_spark.sh

Line 185 in e98dba6

export PYSPARK_PYTHON=python

yes

[masatana]

Okay, Thank you! I'll create a follow-up JIRA ticket.

[JiaLiangC]

I think PYSPARK_PYTHON=python2 is unnecessary. We can simply specify python3 in installation script . Users will set a more specific Python path in their environment settings when they use it.