[infra] Upgrade Python to 3.10.14 in base-builder & base-runner Images

The changes introduced here upgrade Python from 3.8 to 3.10.14 inside
the base-builder and base-runner images.

 ### base-builder changes:

Prior to these changes, base-builder compiled Python 3.8 from source
using sources downloaded from the official release servers at
https://www.python.org/ftp/python/. This updates the compiled version
to 3.10.14 (the latest 3.10 release) instead.

 ### base-runner changes:

Prior to these changes, base-runner installed Python 3.8 from the
default apt repository provided by the Ubuntu 20.04 image it's based
on. These apt repositories do not have a version of Python 3.10
available by default. This updates the base-runner to instead use a
multi-stage build to copy the same Python interpreter compiled by the
base-builder image into the runner image, which ensures both Python
versions remain in-sync while saving build time by re-using a pre-built
version.

 ## Motivation

- Code coverage does not work on Python projects that use Python 3.10+
  syntax, and will not work until this or similar changes are landed
  (see google#11419)
- Upgrading the base-image to use Ubuntu 22.04 (which provides more
  recent Python versions via apt) has been stated as being unlikely to
  happen any time soon (see google#3290)
- Many OSS-Fuzz integrated Python projects no longer support Python 3.8
  and have resorted to implementing ad-hoc workarounds to upgrade to
  newer Python versions, including installing Python from the Dead
  Snakes PPA.
  - This leads to fragmentation and hard to debug issues. Maintenance
    is easier when everyone is using the same version without issue.
- With [Python 3.8 reaching end of life soon (in 2024-10)][python-
  versions-EOL], it is likely that more Python projects will begin
  dropping support for 3.8, further increasing the number of broken
  builds and ad-hoc workarounds.
- Previous attempts at upgrading Python have stalled.

 ## Known & Expected Issues

Several project Dockerfiles and build scripts contain hard coded
references to python3.8 file system paths, and many more have implanted
ad-hoc workarounds to upgrade to newer Python versions than 3.8
(typically 3.9.) Additional changes are required to each of these
projects to ensure they successfully build after this upgrade to Python
3.10.

 ### Fuzz Introspector Caveat

Fuzz Introspector currently uses Python 3.9. While an upgrade to 3.10 is
not expected to introduce any new issues, it was not tested on these
changes and may require additional work.

 ## Possible Areas of Improvement

Using the base-builder image in a multi-stage build to copy the pre-
compiled Python into base-runner is effective, but feels like a
workaround that may be introducing tech debt. A cleaner approach would
be to extract the Python compilation into a discrete base image similar
to how `base-clang` works, and use that as the multi-stage builder in
images that need it.

---

Fixes:
- google#11419

Supersedes:
- google#9532
- google#11420

[python-versions-EOL]: https://devguide.python.org/versions/

infra/base-images/base-builder/Dockerfile

@@ -19,9 +19,9 @@ FROM gcr.io/oss-fuzz-base/base-clang
 COPY install_deps.sh /
 RUN /install_deps.sh && rm /install_deps.sh
 
-# Build and install latest Python 3 (3.8.3).
-ENV PYTHON_VERSION 3.8.3
-RUN export PYTHON_DEPS="\
+# Build and install latest Python 3.10.
+ENV PYTHON_VERSION 3.10.14
+RUN PYTHON_DEPS="\
         zlib1g-dev \
         libncurses5-dev \
         libgdbm-dev \
@@ -39,12 +39,14 @@ RUN export PYTHON_DEPS="\
     tar -xvf Python-$PYTHON_VERSION.tar.xz && \
     cd Python-$PYTHON_VERSION && \
     ./configure --enable-optimizations --enable-shared && \
-    make -j install && \
+    make -j$(nproc) install && \
     ldconfig && \
-    ln -s /usr/bin/python3 /usr/bin/python && \
+    ln -s /usr/local/bin/python3 /usr/local/bin/python && \
     cd .. && \
     rm -r /tmp/Python-$PYTHON_VERSION.tar.xz /tmp/Python-$PYTHON_VERSION && \
-    rm -rf /usr/local/lib/python3.8/test && \
+    rm -rf /usr/local/lib/python${PYTHON_VERSION%.*}/test && \
+    python3 -m ensurepip && \
+    python3 -m pip install --upgrade pip && \
     apt-get remove -y $PYTHON_DEPS # https://github.com/google/oss-fuzz/issues/3888
 
 # Install six for Bazel rules.
@@ -177,4 +179,4 @@ COPY llvmsymbol.diff $SRC
 COPY detect_repo.py /opt/cifuzz/
 COPY bazel.bazelrc /root/.bazelrc
 
-CMD ["compile"]
+CMD ["compile"]

infra/base-images/base-runner/Dockerfile

@@ -24,6 +24,11 @@ RUN cargo install rustfilt
 # Using multi-stage build to copy some LLVM binaries needed in the runner image.
 FROM gcr.io/oss-fuzz-base/base-clang AS base-clang
 
+# The base builder image compiles a specific Python version. Using a multi-stage build
+# to copy that same Python interpreter into the runner image saves build time and keeps
+# the Python versions in sync.
+FROM gcr.io/oss-fuzz-base/base-builder AS base-builder
+
 # Real image that will be used later.
 FROM gcr.io/oss-fuzz-base/base-image
 
@@ -35,6 +40,18 @@ COPY --from=base-clang /usr/local/bin/llvm-cov \
      /usr/local/bin/llvm-symbolizer \
      /usr/local/bin/
 
+# Copy the pre-compiled Python binaries and libraries
+COPY --from=base-builder /usr/local/bin/python3.10 /usr/local/bin/python3.10
+COPY --from=base-builder /usr/local/lib/libpython3.10.so.1.0 /usr/local/lib/libpython3.10.so.1.0
+COPY --from=base-builder /usr/local/include/python3.10 /usr/local/include/python3.10
+COPY --from=base-builder /usr/local/lib/python3.10 /usr/local/lib/python3.10
+COPY --from=base-builder /usr/local/bin/pip3 /usr/local/bin/pip3
+
+# Create symbolic links to ensure compatibility
+RUN ldconfig && \
+    ln -s /usr/local/bin/python3.10 /usr/local/bin/python3 && \
+    ln -s /usr/local/bin/python3.10 /usr/local/bin/python
+
 COPY install_deps.sh /
 RUN /install_deps.sh && rm /install_deps.sh
 

infra/base-images/base-runner/install_deps.sh

@@ -20,12 +20,10 @@
 apt-get update && apt-get install -y \
     binutils \
     file \
+    ca-certificates \
     fonts-dejavu \
     git \
     libcap2 \
-    python3 \
-    python3-pip \
-    python3-setuptools \
     rsync \
     unzip \
     wget \