[CI][C++] Force CMake to find Python2

[BewareMyPower]

Fixes #9682

Motivation

Currently, ci-cpp-tests uses pulsar-build image that is from ubuntu:16.04 to build C++/Python client. The image uses libboost-all-dev for CMake to find boost dependencies. However, the Boost.Python library from Ubuntu 16.04's apt source only supports Python 2.

Modifications

• Specifying PYTHON_INCLUDE_DIR and PYTHON_LIBRARY could indicate the installation of Python to use. Since the pulsar-build image only contains Python binary but not the Python2 library (libpython2.7so), this PR installs libpython-dev to setup the Python2 library. Otherwise, CMake would still find the Python3 library(libpython3.5.so).
• Remove redundant C++ client dependencies like libjsoncpp-dev and replace libboost-all-dev with the specific libboost-xxx-dev.

Verifying this change

• Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

[BewareMyPower]

After discussing with @eolivelli in #9684 , instead of removing Python2 which may need a lot of work, this PR just keeps the original CI and lets CMake to find Python2 only in CI.

Since there's no way to verify pulsar-build image yet, I still use the image that was built by myself first. After the CI passed, I think committers that have permissions to apachepulsar's dockerhub could upload the new image and run CI again.

[BewareMyPower]

It looks like pip install setuptools will fail

Collecting setuptools>=34.0.0 (from apache-bookkeeper-client>=4.9.2->pulsar-client==2.8.0)
  Downloading https://files.pythonhosted.org/packages/12/68/95515eaff788370246dac534830ea9ccb0758e921ac9e9041996026ecaf2/setuptools-53.0.0.tar.gz (2.1MB)
    100% |################################| 2.1MB 502kB/s 
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "setuptools/__init__.py", line 16, in <module>
        import setuptools.version
      File "setuptools/version.py", line 1, in <module>
        import pkg_resources
      File "pkg_resources/__init__.py", line 1367
        raise SyntaxError(e) from e
                                ^

It tried to download setuptools 53.0.0 but it doesn't support Python2. I saw the successfully CI log is

Requirement already satisfied: setuptools>=34.0.0 in /usr/local/lib/python2.7/dist-packages (from apache-bookkeeper-client>=4.9.2->pulsar-client==2.8.0) (44.1.1)

I found the cause is the pip version of my image is 8.1.1 but the pip version of apachepulsar/pulsar-build is 20.3.4. I'll solve this problem.

[eolivelli]

what about installing pip 2.7 with this command ?

curl https://bootstrap.pypa.io/2.7/get-pip.py | python

[BewareMyPower]

@eolivelli Yeah, I've already tested successfully in my local environment, just waiting for docker image uploaded.

[BewareMyPower]

@zymap The CPP, Python Tests have passed. I think you can upload a new image now.

[eolivelli]

LGTM
thank you!

[merlimat]

@BewareMyPower This should also work with the current image on DockerHub, right?

Just saying that because we could merge this PR, wait few days and then push the image, so that we minimize the chances of having PRs blocked on CI because they were created on top of a version of master that didn't include this PR yet.

[merlimat]

Uhm, actually, we should just call this image with a different tag... I'll push the image

[BewareMyPower]

Thanks @merlimat

[BewareMyPower]

/pulsarbot run-failure-checks

[eolivelli]

/pulsarbot run-failure-checks

[eolivelli]

@aahmed-se @BewareMyPower @merlimat
QA tests passed.
Can we merge this patch and push the new image ?
Thank you very much !

[BewareMyPower]

@eolivelli It looks like @merlimat has already pushed a new image apachepulsar/pulsar-build:ubuntu-16.04-py2 which has a new tag ubuntu-16.04-py2. See commit 0726016. The current CI is based on the new image.

[eolivelli]

@BewareMyPower thanks for your information.

and thanks to @merlimat for pushing the new image.

I will start to run CI jobs in JDK11 on my private fork and see what happens.

I hope we can merge this patch soon and keep the master branch up to date

[merlimat]

I just did the basic versioning with -py2 here, we can do -py3 for the other changes. In future we should prob have a better convention to tag these images.