docker manifest inspect currently failing on some GHCR images

[pedrolamas]

Select Topic Area

Bug

Body

Until yesterday, I was using docker manifest inspect ghcr.io/USER/CONTAINER:TAG to check if a specific container tag already existing, but that now seems to have stopped working...

Specifically, my docker-klipper-simulavr image doesn't even show for "latest" tag:

> docker manifest inspect ghcr.io/pedrolamas/docker-klipper-simulavr:latest
no such manifest: ghcr.io/pedrolamas/docker-klipper-simulavr:latest

The above now just returns "no such manifest: ghcr.io/pedrolamas/docker-klipper-simulavr:latest" when the image is actually there and properly tagged with "latest"!

docker pull ghcr.io/pedrolamas/docker-klipper-simulavr:latest works fine and I can even access it directly via browser, but sometimes I do get a "tag does not exist" red warning from GitHub when I do that...

Any idea of what is going on or where should I report issues with GHCR ?

[ChristopherHX]

No idea what is going on, however there is an alternative docker tool to do what you want

You can use docker buildx imagetools inspect ghcr.io/pedrolamas/docker-klipper-simulavr:latest it just works fine.

I can confirm that docker manifest doesn't find your image on my machine.

I prefer docker buildx imagetools for publishing multiarch images from a local docker registry over docker manifest (it refuses to copy images from the source to the target registry).

[pedrolamas]

Indeed, I do see the "attestation-manifest" and I have no idea what it is, but that does seem to be "a thing", considering it also shows on the builx imagetools inspect docs

Comparing your build workflow with mine, the only major difference is that I am just using Docker GitHub Actions (like docker/build-push-action@v3) instead of docker commands...

Makes me wonder if there has been an update on these actions that are now causing issues...

[pedrolamas]

I've found what that extra manifest it: https://docs.docker.com/build/attestations/attestation-storage/

[pedrolamas]

I have a feeling I have found the source of the problem: 2 weeks ago there the build-push-action action released v3.3.0 that seems to have added a new "attestation" that is not fully compatible with GHCR!

I've found this report or issues with this change, which I think it might be related to the whole situation.

[djmaze]

Ah, so people should set provenance: false when building their images on Github as shown here.