Cannot partially mirror fat manifests / manifest lists

[Jamstah]

I missed the #18216 autoclose, so opening a new one.

This proposal is heavily copied from distribution/distribution#3628. I believe this proposal depends on that one. This is aka Sparse Manifests.

Is your feature request related to a problem? Please describe.

This issue covers an image size concern based on the use of image indexes.

There are three things that combine here:

• Image indexes provide platform architecture portability with low friction to clients, they list references by digest which means you can be sure you're getting the expected content.
• Signing enables trust in images, and signing an index is a good way to say "these are definitely the platform images you want". However, once you sign the image index, you can't change the references without invalidating the signature.
• The distribution code will validate image indexes on push to ensure the referenced platform specific image manifests (and therefore their blobs) exist in the registry.

Putting these three things together, there is no way to copy a subset of architectures of an index to a mirror without losing the signature, changing the digest of the index, or losing the index and having to pull platform images directly. Forcing the user to copy all architectures regardless of the ones they will be running within their environment makes the mirror process longer, uses more storage, uses more network, and increases the load on vulnerability scan within the organisation, especially when we're talking about 100s of images.

Describe the solution you'd like
I'd like to avoid these pitfalls by making it possible to push an index even if its references are missing, if the registry admin configures it that way.

My vested interest in this is that I work for IBM developing cloud paks. Our customers use multiple different architectures, but customers don't want to have to mirror every architecture to get the images they want into their restricted network environments. As developers, we want to use image indexes to simplify deployments, support multi architecture k8s clusters, and sign everything to secure deployments, so would prefer a way for customers to mirror partial image indexes over having to not use image indexes at all.

Describe the main design/architecture of your solution
I have submitted a PR to distribution/distribution, see: distribution/distribution#3628

Describe the development plan you've considered
I have submitted a PR to distribution/distribution, the work for harbor would be to update to a newer level and add the configuration options, along with documentation.

Additional context
This is related to a discussion I started on the opencontainers list. The result of this discussion was to make it more clear in the spec that it is perfectly valid behaviour for registries to not validate the existence of references platform specific images:

• https://groups.google.com/a/opencontainers.org/g/dev/c/Uw8xdBOr444
• opencontainers/distribution-spec@3c2316e

I have contributed changes to skopeo to enable mirroring of image indexes without mirroring the underlying platforms:

• Add an option to allow copying image indexes alone containers/skopeo#1511

I have contributed changes to containers/image to improve error messages where image indexes are missing platform specific images:

• Improve errors messages when image missing from list containers/image#1550