Use Docker API instead of Docker CLI

[mdonoughe]

This uses the Docker API via the Bollard crate instead of calling out to the Docker CLI and parsing JSON output.

Why do this?

1. Bollard handles the data models and parsing.
2. Docker CLI does not need to be installed.
3. When making the API calls through Bollard, the error codes are available to Docuum.

The main motivation behind this is so the call to remove an image can tell when the image could not be deleted due to a conflict (eg dependent images still exist) and then be reasonably sure that the amount of used space has not decreased to an acceptable level so the amount of free space does not need to be recalculated before attempting the next deletion. I estimate the change to skip recalculating space usage made Docuum run 10x faster catching up on a build agent that was used for builds that produced many docker layers over many multistage builds, which when not catching up could still reduce impact on builds that are triggering Docuum to clean up.

That behavior change is not part of this PR. I want to keep this level of refactoring separate from behavior changes.

This is the same code as last time, with some changes from https://github.com/mac-chaffee/docuum/tree/BACKUP2, and an updated version of Bollard which should not have the same problems with BuildKit.

Status: Ready

Fixes: N/A

[mdonoughe]

Should the Docker image use distroless instead of buster-slim? The resulting image is only 28.4MB instead of 78.8MB.

[stepchowfun]

Distroless seems fine. Alpine is another option (this seems to indicate that Alpine is somehow smaller than distroless). Another option is to switch to musl and produce a fully static binary with no dependencies, and then use FROM scratch. I have no strong preferences on this, as long as the image is a "reasonable" size (I agree that 78.8 MB seems too large).

I'll review this as soon as I can. Thanks for bringing it back! I know it's mostly the same as before, so I just need to look at the parts that have changed due to #73.

[mdonoughe]

I did a release build using x86_64-unknown-linux-musl and the resulting Docker image was 9.48 MB, but I needed to install an additional Rust target and Ubuntu package. Distroless is larger because it contains libc and libstdc++ and libopenssl.

mdonoughe@DESKTOP-4KQENMJ:~/distroless-cc$ dust
 1.0M       ┌── libm-2.24.so         │                                                           ▒▒▒▒▒▒▒▒▒▒████ │   5%
 1.6M       ├── libc-2.24.so         │                                                           ▒▒▒▒▒▒▒▒██████ │   8%
 3.9M     ┌─┴ x86_64-linux-gnu       │                                                           ██████████████ │  18%
 3.9M   ┌─┴ lib                      │                                                           ██████████████ │  18%
 640K   │   ┌── openssl              │               ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░███ │   3%
 652K   │ ┌─┴ bin                    │               ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░███ │   3%
 988K   │ │ ┌── doc                  │               ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░▒▒▒▒▒▒▒▒▒▒▒▒▒▒████ │   5%
 524K   │ │ │ ┌── America            │               ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░▒▒▒▒▒▒▓▓▓▓▓▓▓▓▓▓██ │   2%
 536K   │ │ │ │ ┌── America          │               ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░▒▒▒▒▒▒▓▓▓▓▓▓██████ │   2%
 1.6M   │ │ │ ├─┴ right              │               ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░▒▒▒▒▒▒▓▓▓▓▓▓██████ │   8%
 3.4M   │ │ ├─┴ zoneinfo             │               ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░▒▒▒▒▒▒████████████ │  16%
 5.2M   │ ├─┴ share                  │               ░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░██████████████████ │  24%
 436K   │ │   ┌── libssl.so.1.1      │               ░░░░░░░░░░░░░░░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓██ │   2%
 1.5M   │ │   ├── libstdc++.so.6.0.22│               ░░░░░░░░░░░░░░░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓██████ │   7%
 2.6M   │ │   ├── libcrypto.so.1.1   │               ░░░░░░░░░░░░░░░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓█████████ │  12%
 460K   │ │   │ ┌── libCNS.so        │               ░░░░░░░░░░░░░░░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓██████████████████████ │   2%
 6.3M   │ │   ├─┴ gconv              │               ░░░░░░░░░░░░░░░░░░░░▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓▓██████████████████████ │  30%
  11M   │ │ ┌─┴ x86_64-linux-gnu     │               ░░░░░░░░░░░░░░░░░░░░██████████████████████████████████████ │  52%
  11M   │ ├─┴ lib                    │               ░░░░░░░░░░░░░░░░░░░░██████████████████████████████████████ │  52%
  16M   ├─┴ usr                      │               ██████████████████████████████████████████████████████████ │  79%
  21M ┌─┴ .                          │█████████████████████████████████████████████████████████████████████████ │ 100%

[stepchowfun]

LGTM. I'll release this now, and any changes to the Docker base image can be made separately as people see fit.

[stepchowfun]

@mac-chaffee Now that this has landed, would you like to try out the latest version of Docuum (v0.12.0) and confirm that the issue you noticed before no longer occurs?

[mac-chaffee]

@stepchowfun Looks good! And it's a lot faster too, I guess due to not having to spawn so many docker CLI processes! Excellent work @mdonoughe !

$ docker system df
TYPE                TOTAL               ACTIVE              SIZE                RECLAIMABLE
Images              64                  12                  17.98GB             13.9GB (77%)
Containers          12                  11                  13.41MB             516B (0%)
Local Volumes       10                  9                   247.9MB             0B (0%)
Build Cache         1336                0                   41.8GB              41.8GB

$ docker run \
>   --detach \
>   --init \
>   --rm \
>   --name docuum \
>   --volume /var/run/docker.sock:/var/run/docker.sock \
>   --volume docuum:/root \
>   stephanmisc/docuum --threshold '15 GB'
3ce151ef050a97b27a00b5a3278583477d4f8d59eea5c71be5dc0d4a1175811a

$ docker logs -f 3ce151ef050a97b27a00b5a3278583477d4f8d59eea5c71be5dc0d4a1175811a
[2020-08-06 14:55:42 +00:00 INFO] Waking up…
[2020-08-06 14:55:42 +00:00 INFO] Docker images are currently using `17.98 GB` but the limit is `15.00 GB`. Some images will be deleted.
[2020-08-06 14:55:43 +00:00 INFO] Deleting image `sha256:8a23297b2305aee7e6259b2d9ae5e67745cae3c4b695d7858260cd69e1f96ef7`…
[2020-08-06 14:55:43 +00:00 ERROR] Unable to delete image `sha256:8a23297b2305aee7e6259b2d9ae5e67745cae3c4b695d7858260cd69e1f96ef7`: Error { inner:

API responded with a 409 conflict: {"message":"conflict: unable to delete 8a23297b2305 (cannot be forced) - image has dependent child images"}
 }.
...<truncated>...
[2020-08-06 14:56:32 +00:00 INFO] Deleting image `sha256:9f476216c4f2567ba8b0cfae57545c16a67a18989340ec8b252e0f136dd7e8cc`…
[2020-08-06 14:56:32 +00:00 INFO] Docker images are now using `14.62 GB`, which is within the limit of `15.00 GB`.
[2020-08-06 14:56:32 +00:00 INFO] Going back to sleep…