Test Docker Manifest Support

[Ichimonji10]

Pulp 2.14 adds support for Docker manifest lists. This change is significant, and it really needs to be tested before 2.14 is released. Some of the testing for this new feature can be automated, but given the lack of Docker clients on our automated systems, some of the testing will need to be manual.

A docker manifest list is a list of docker manifests, and a docker manifest describes the layers of a docker image. A typical docker manifest list will list out one docker manifest per supported architecture, where supported architectures are values like amd64-linux or arm32v7. In addition, there's two versions of docker manifests: schema v2 and schema v1.

New docker clients can natively understand docker manifest lists. When a new docker client contacts the docker hub (or a clone like Pulp + Crane), it will express its support for this new format by sending certain HTTP headers saying so. When an older docker client contacts the docker hub (or a clone), the hub must be capable of downgrading its response on-the-fly. Pulp + Crane do this by pre-emptively contacting the docker hub and fetching all the content that each generation of docker clients might want.

We need to create automated tests that do the following:

• Create a Docker repository, and let it sync content from a repository that has a manifest list with entries for more than one architecture.
• Create a Docker repository, and let it sync content from a repository that has a manifest list with no entry for the amd64-linux architecture.

For each repository, we need the following tests:

• Pretend to be a modern Docker client, and get (or at least list) the contents of the repository.
• Pretend to be an older Docker client, and get (or at least list) the contents of the repository.
• Pretend to be an even older Docker client, and get (or at least list) the contents of the repository.

We also need to manually test this new feature by installing and configuring the official Docker client and ensuring that it can docker pull from each repository.

To fully understand the testing that needs to be done, please read https://docs.docker.com/registry/spec/manifest-v2-2/#backward-compatibility

The following Docker repositories may be useful for testing:

• https://hub.docker.com/r/dmage/manifest-list-test/tags/
• https://hub.docker.com/r/dmage/busybox/tags/

Related to: #555

[preethit]

Adding issues that may be related to this
https://pulp.plan.io/issues/2924
https://pulp.plan.io/issues/2920

[Ichimonji10]

Also see:

• https://pulp.plan.io/issues/2384
• https://pulp.plan.io/issues/2385