Provide a smaller Pulsar docker image without unnecessary development packages

[lhotari]

Is your enhancement request related to a problem? Please describe.

The pulsar docker image installs a lot of packages that are unnecessary.
For example, installing the tk-dev package pulls in many dependencies

The following NEW packages will be installed:
  binutils binutils-common binutils-x86-64-linux-gnu build-essential cpp cpp-9 dirmngr dpkg-dev fakeroot fontconfig-config fonts-dejavu-core g++ g++-9 gcc gcc-9 gcc-9-base gnupg
  gnupg-l10n gnupg-utils gpg gpg-agent gpg-wks-client gpg-wks-server gpgconf gpgsm libalgorithm-diff-perl libalgorithm-diff-xs-perl libalgorithm-merge-perl libasan5
  libasn1-8-heimdal libassuan0 libatomic1 libbinutils libbsd0 libc-dev-bin libc6-dev libcc1-0 libcrypt-dev libctf-nobfd0 libctf0 libdpkg-perl libdrm-amdgpu1 libdrm-common
  libdrm-intel1 libdrm-nouveau2 libdrm-radeon1 libdrm2 libedit2 libelf1 libexpat1 libexpat1-dev libfakeroot libfile-fcntllock-perl libfontconfig1 libfontconfig1-dev libfontenc1
  libfreetype-dev libfreetype6 libfreetype6-dev libgcc-9-dev libgdbm-compat4 libgdbm6 libgl1 libgl1-mesa-dri libglapi-mesa libglib2.0-0 libglib2.0-data libglvnd0 libglx-mesa0
  libglx0 libgomp1 libgssapi3-heimdal libhcrypto4-heimdal libheimbase1-heimdal libheimntlm0-heimdal libhx509-5-heimdal libice-dev libice6 libicu66 libisl22 libitm1
  libkrb5-26-heimdal libksba8 libldap-2.4-2 libldap-common libllvm12 liblocale-gettext-perl liblsan0 libmpc3 libmpfr6 libnpth0 libpciaccess0 libperl5.30 libpng-dev libpng-tools
  libpng16-16 libpthread-stubs0-dev libquadmath0 libreadline8 libroken18-heimdal libsasl2-2 libsasl2-modules libsasl2-modules-db libsensors-config libsensors5 libsm-dev libsm6
  libsqlite3-0 libssl1.1 libstdc++-9-dev libtcl8.6 libtk8.6 libtsan0 libubsan1 libutempter0 libvulkan1 libwayland-client0 libwind0-heimdal libx11-6 libx11-data libx11-dev
  libx11-xcb1 libxau-dev libxau6 libxaw7 libxcb-dri2-0 libxcb-dri3-0 libxcb-glx0 libxcb-present0 libxcb-randr0 libxcb-shape0 libxcb-shm0 libxcb-sync1 libxcb-xfixes0 libxcb1
  libxcb1-dev libxcomposite1 libxdmcp-dev libxdmcp6 libxext-dev libxext6 libxfixes3 libxft-dev libxft2 libxi6 libxinerama1 libxkbfile1 libxml2 libxmu6 libxmuu1 libxpm4 libxrandr2
  libxrender-dev libxrender1 libxshmfence1 libxss-dev libxss1 libxt-dev libxt6 libxtst6 libxv1 libxxf86dga1 libxxf86vm1 linux-libc-dev make manpages manpages-dev
  mesa-vulkan-drivers netbase patch perl perl-modules-5.30 pinentry-curses pkg-config readline-common shared-mime-info tcl tcl-dev tcl8.6 tcl8.6-dev tk tk-dev tk8.6 tk8.6-dev
  tzdata ucf uuid-dev x11-common x11-utils x11proto-core-dev x11proto-dev x11proto-scrnsaver-dev x11proto-xext-dev xbitmaps xdg-user-dirs xorg-sgml-doctools xterm xtrans-dev
  xz-utils zlib1g-dev

I used these commands to find out what pulls in the enourmous amount of packages:

docker run --rm -it ubuntu:20.04 bash
apt-get update
for i in openjdk-11-jdk-headless netcat dnsutils less procps iputils-ping python3 python3-dev python3-setuptools python3-yaml python3-kazoo libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev                  curl; do echo $i; apt-get install -s $i; done

Describe the solution you'd like

The Pulsar docker image should not contain development tools for compiling.

[hpvd]

+1 on this!
goes into the same direction as b) in #8815 (comment) thinks...

[lhotari]

+1 on this! goes into the same direction as b) in #8815 (comment) thinks...

Thank you @hpvd. You might be also interested in the work @michaeljmarshall has done for #11269

[hpvd]

@lhotari
would you mind

• pointing to a subdirectory of https://hub.docker.com/u/apachepulsar where one can obtain the smaller image and
• cross-check the MBs saved in size (before and after this change - should be arround v2.10)?

Do the savings meet your expectations?

Do we have to make any changes in text or link targets in doc on https://pulsar.apache.org/docs/next/deploy-docker ?

[lhotari]

@lhotari would you mind pointing to a subdirectory of https://hub.docker.com/u/apachepulsar where one can obtain the smaller image and cross-check the MBs saved in size (before and after this change - should be arround v2.10)?

@hpvd I think the simplest way to get a comparison of #14093 changes would be to build a docker images before and after locally.

Do the savings meet your expectations?

yes, for the particular changes. However, I think that Pulsar lacks a proper slim image, but that's a topic for a separate discussion.

Do we have to make any changes in text or link targets in doc on https://pulsar.apache.org/docs/next/deploy-docker ?

That document looks very much outdated. It doesn't make sense to me.

[hpvd]

thanks for getting back that fast!

ps: filed an issue for doc update #17774

[lhotari]

thanks for getting back that fast!

ps: filed an issue for doc update #17774

Thank you @hpvd

[michaeljmarshall]

I've found dive to be a helpful tool for inspecting docker images.