breaking: hub image ubuntu->debian, py38->py39, build-essential removed, --build-arg PIP_OVERRIDES=... removed, images/hub/dependencies removed

[consideRatio]

I've trialed a switch from ubuntu:20.04 to python:3.9-slim-bullseye and think this is a breaking change we should do. Closes #2731. Some of the changes in this PR are changes that could be done with ubuntu:20.04 as well.

Image size

Using dive:

• Old image: 623 MB
• New image: 437 MB (70% of old image)

About removed apt packages

• libpq-dev doesn't have to be installed any more as there is a pip wheel for aarch64 now making it not needed.
• build-essential isn't needed, but removing it makes people who install packages later that require gcc etc need to install it.
• python3* is removed as the base image provides python for us.

About pycurl apt package dependencies

It seems that we can get openssl as a backend for pycurl (introduced in #1185) by referencing libcurl4 without needing to install libssl-dev and libcurl4-openssl-dev.

If I test to run python -c "import pycurl; print(pycurl.version_info())" in the 1.2.0 image and compare with the output from this new Dockerfile we see this.

# pycurl.version_info() from this Dockerfile
(7, '7.74.0', 477696, 'x86_64-pc-linux-gnu', 29345693, 'OpenSSL/1.1.1n', 0, '1.2.11', ('dict', 'file', 'ftp', 'ftps', 'gopher', 'http', 'https', 'imap', 'imaps', 'ldap', 'ldaps', 'mqtt', 'pop3', 'pop3s', 'rtmp', 'rtsp', 'scp', 'sftp', 'smb', 'smbs', 'smtp', 'smtps', 'telnet', 'tftp'), None, 0, '2.3.0')
# pycurl.version_info() from the hub image version 1.2.0
(5, '7.68.0', 476160, 'x86_64-pc-linux-gnu', 12568477, 'OpenSSL/1.1.1f', 0, '1.2.11', ('dict', 'file', 'ftp', 'ftps', 'gopher', 'http', 'https', 'imap', 'imaps', 'ldap', 'ldaps', 'pop3', 'pop3s', 'rtmp', 'rtsp', 'scp', 'sftp', 'smb', 'smbs', 'smtp', 'smtps', 'telnet', 'tftp'), None, 0, '2.2.0')

About removed --build-arg PIP_OVERRIDES

This was introduced by me in #1948. I think it should be removed now. We have requirements.txt to allow anyone pin whatever version if the repo is cloned the repo, and at worst someone would have to do a sed replace operation to the in the requirements.txt file or append a line if they have automation to make their own build of the hub image.

Above removed images/hub/dependencies script

This script was removed for binderhub in jupyterhub/binderhub#1433, but not for z2jh because of how we passed --build-arg with PIP_OVERRIDES something we stopped with.

The script is no longer needed, but instead we can simply do this and we are done. This script works both locally and in a GitHub CI environment as well, which makes us able to use automation to update this.

cd images/hub
docker run --rm \
    --env=CUSTOM_COMPILE_COMMAND="see README.md" \
    --volume=$PWD:/io \
    --workdir=/io \
    --user=root \
    python:3.9-bullseye \
    sh -c 'pip install pip-tools==6.* && pip-compile --upgrade'

[minrk]

Very nice!

[yuvipanda]

yay! I also appreciate the addition of tini.