skopeo should be able to return be the catalog from a docker-distribution based registry.

[rhatdan]

A User want to build tooling to keep multiple container registries in sync using skopeo.
In order to script this he needs to have a list of all container images at the source registry. This is stored as a catalog and should be able to be retrieved using skope catalog command.

Add a new command catalog, which will bring back the entire registry catalog file in json format.

skopeo catalog docker:registry.access.redhat.com

Should output the catalog to stdout.

[runcom]

/cc @mtrmac

[mtrmac]

• This is necessary but not sufficient for full sync because it only lists repositories, not images in these repositories. We would also need Can not list Tags on Repo when there is no latest tag #276 to list tags, and I don’t think there is any way to list untagged images (which are still accessible using the @digest notation).

• If this is supposed to be a generic command, we need to think about what the transport-independent semantics is (e.g. “what should skopeo catalog dir:/usr output?”). Perhaps we can avoid that and make a docker-specific hack for now, skopeo docker-datalog $hostname or something like that.

[so0k]

until skopeo has this, you can do this for Docker specific registries as follows

curl -su $DOCKER_USERNAME:$DOCKER_PASSWORD https://$DOCKER_REGISTRY/v2/_catalog | jq .
curl -su $DOCKER_USERNAME:$DOCKER_PASSWORD https://$DOCKER_REGISTRY/v2/$IMAGE_NAME/tags/list | jq .

or in a single line (generate array of all images with their tags) using curl,xargs and jq:

curl -su $DOCKER_USERNAME:$DOCKER_PASSWORD https://$DOCKER_REGISTRY/v2/_catalog \
  | jq -r .repositories[] \
  | xargs -P 8 -I'{}' curl -su $DOCKER_USERNAME:$DOCKER_PASSWORD https://$DOCKER_REGISTRY/v2/{}/tags/list \
  | jq --slurp .

with xargs -P 1:

real    0m11.673s
user    0m1.251s
sys     0m0.792s

with xargs -P 8:

real    0m5.543s
user    0m1.395s
sys     0m0.832s

[rhatdan]

@mtrmac @vrothberg Is this something we should pursue? Should we look at interns working on this? Do you think it is important?

[vrothberg]

I am taking a shot and close it in favor of the skopeo-sync PR #524.

[mtrmac]

skopeo sync won’t use the catalog to list all repositories.

The existing podman search can be used for this, e.g. podman search registry.fedoraproject.org/ (but that registry reasonably limits the number of results, so it is only marginally useful.)

Considering that a sane registry server rarely support returning the full catalog, and that none of this can deal with untagged images, yes, let’s close.