[WIP] docker-py: use pip to install dependencies

[thaJeztah]

follow-up to #38524

This makes sure that we have all the requirements installed and that they have the right version.

Moved the installation earlier in the Dockerfile, as docker-py is not so frequently updated in this repository, so optimizing for caching.

Removed all the apt-packages that are not needed.

[codecov]

Codecov Report

Merging #38526 into master will increase coverage by 0.07%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #38526      +/-   ##
==========================================
+ Coverage   36.64%   36.71%   +0.07%     
==========================================
  Files         608      608              
  Lines       45173    45230      +57     
==========================================
+ Hits        16553    16608      +55     
+ Misses      26339    26333       -6     
- Partials     2281     2289       +8

[olljanat]

PowerPC failure looks like related.

[olljanat]

(reserved for my derek commands)

[cpuguy83]

Nice, I wonder if we can go even further and install docker-py deps into their own image and then COPY them in at the end.

pip install --target would allow us to customize the path at least or can just use the default path.

[kolyshkin]

I would argue that using distro packages should be preferred to using pip -- distro packages are faster to install and (supposedly?) better maintained so more stable. The current approach is to install as much as we can from the distro packages, and use pip for the rest.

The only downside I see is packages to be installed are spread between two places in Dockerfile, and it's not bad enough per se to switch to pip entirely. @thaJeztah perhaps I'm missing something -- what is your motivation to prefer pip to apt install?

[thaJeztah]

Closing this in favour of #39068