Reduce docker image size for Java 17

[yihua]

Task Description

The docker image size is larger for Java 17 build (./build_docker_images.sh --hadoop-version 3.4.0 --spark-version 4.0.1 --hive-version 3.1.3 --multi-arch). We should see if there's way to reduce the image size.

E.g., https://hub.docker.com/r/apachehudi/hudi-hadoop_3.4.0-base/tags

Task Type

Test enhancement

Related Issues

No response

[kartikeyaagrawal]

Image Size Reductions (base_java17)

Starting size: ~3.56 GB

Changes & Savings

• JDK → JRE base image
  17-jdk → 17-jre-jammy
  → ~200 MB saved
  Removes compiler toolchain since the container only runs Hadoop.

• Python: source build → apt install
  python3-minimal + python3-pip (in spark_base)
  → ~600 MB saved
  Avoids installing build-essential, dev headers, and compiling CPython with --enable-optimizations.

• Multi-stage build optimizations

  • --no-install-recommends
  • Cleaned apt lists
  • Dropped unused wget and .asc downloads
    → ~100 MB saved
    Builder stage isolates curl/ca-certs and Hadoop tarball.

---

Total Impact

~3.56 GB → ~2.58 GB
→ ~900 MB saved (~27% reduction)

Can I raise the pr for the above optimization ? Or do we require more optimization?

[yihua]

@kartikeyaagrawal have you tried to build the image with ./build_docker_images.sh --hadoop-version 3.4.0 --spark-version 4.0.1 --hive-version 3.1.3 and see the size difference before and after? The current size is 1.1GB (https://hub.docker.com/r/apachehudi/hudi-hadoop_3.4.0-base) whereas older hudi hadoop base image is 465MB (https://hub.docker.com/r/apachehudi/hudi-hadoop_2.8.4-base), which is different from what is being claimed here (3.56GB).