Preserve annotations in OCI images

[owtaylor]

If you copy an OCI image to a docker registry and copy it back, then the annotations in the image should be preserved. For example, an OCI image generated by Flatpak can be found at:

https://people.gnome.org/~otaylor/example-flatpak-oci-20170621.tar.gz

There are a couple of incompatibilities with skopeo in that image:

• It uses org.opencontainers.ref.name rather than org.opencontainers.image.ref.name (probably an older spec version)
• The ref name it uses "app/org.gnome.eog/x86_64/master" isn't accepted by skopeo even though it looks OK by the spec

But if you fix those manually, and copy to a docker registry and copy back (I used a local registry set up with https://hub.docker.com/_/registry/), then starting with an index.json:

{
    "schemaVersion" : 2,
    "mediaType" : "application/vnd.oci.image.index.v1+json",
    "manifests" : [
    {
       "mediaType" : "application/vnd.oci.image.manifest.v1+json",
       "digest" : "sha256:0c02d6e94d7c223b4f1e67ef0a194092a67f9115293f4a5e8e5f47305057dede",
       "size" : 2067,
       "annotations" : {
           "org.flatpak.installed-size" : "4427776",
           "org.flatpak.metadata" : "[Application]\nname=org.gnome.eog\nruntime=org.fedoraproject.Platform/x86_64/26\nsdk=org.fedoraproject.Platform/x86_64/26\ncommand=eog\n\n[Context]\nshared=ipc;\nsockets=x11;wayland;session-bus;\nfilesystems=xdg-run/dconf;host;~/.config/dconf:ro;\n\n[Session Bus Policy]\nca.desrt.dconf=talk\n\n[Environment]\nDCONF_USER_CONFIG_DIR=.config/dconf\n",
            "org.opencontainers.image.ref.name" : "latest",
            "org.flatpak.download-size" : "1866664"
      }
    }
  ]
}

It ends up as:

{
"schemaVersion":2,
"manifests":[
    {
	"mediaType":"application/vnd.oci.image.manifest.v1+json",
	"digest":"sha256:0d7b52181d395460b6dbc5d336d147618424d804f8a6047e136e71ce0e181700",
	"size":348,
	"annotations":{
	    "org.opencontainers.image.ref.name":"latest"
	},"
	"platform":{
	    "architecture": "amd64",
	    "os":"linux"
	}
    }
]
}

And similarly, the annotations are also dropped from the manifest that the index points to.

[mtrmac]

WRT annotations in the manifest, I think that’s expected: AFAIK none of the existing docker/distribution registries support OCI images directly, so the OCI manifest is converted into a schema2 manifest. (To be fair, right now even if a registry did support OCI, the code wouldn't try to use it; we try only schema2/schema1.) That manifest schema does not support annotations, so they are dropped during the conversion. Copying the image back to OCI/layout converts the manifest to OCI again, but the annotations are gone by then.

The annotations in the index are not even treated as a part of the OCI image in our current implementation, the “concept” of an image as currently implemented does not have any external annotations. The OCI transport only uses the index to look up a manifest file by name. (I am not really sure what the semantics of index vs. image vs. annotation is supposed to be, perhaps @runcom knows. OCI annotations just seem horribly overgeneralized to me either way.)

[mtrmac]

It uses org.opencontainers.ref.name rather than org.opencontainers.image.ref.name (probably an older spec version)

Yes, containers/image#278 .

The ref name it uses "app/org.gnome.eog/x86_64/master" isn't accepted by skopeo even though it looks OK by the spec

This seems to be a bug , please file it at https://github.com/containers/image . (We may have to break the transport syntax now that an arbitrary number of : is allowed in the .tag :( )

[owtaylor]

For the purposes of Flatpak, the annotations in the index.json should not needed be if the manifest is preserved. The current code might look in the index.json for them, but it can be adjusted. Does it might make sense to copy the annotations back from the manifest to the index when creating one for an oci: destination? That seems to be a question for an image spec expert. (@vbatts?)

distribution/distribution#2076 is a PR to add OCI manifest support to the docker registry. It is apparently blocked on finalizing the image spec, and also may be out of date with respect to the current image spec, but if the issue is registry-side support would hopefully allow resolving this when completed.

(@runcom asked me to file this issue to track things)

[mtrmac]

(If anyone wants to experiment with OCI support in a docker/distribution registry, it should be necessary to only add the OCI manifest type to docker_image_dest.go:manifestMIMETypes; then a round-trip of an OCI image to/from the server should preserve the original manifest unmodified, or at most compress the layers if they are not compressed.)

[vbatts]

@owtaylor preserving annotations ought to be done, but as for when they get into the index, that could vary for use-case. Presently the context of different annotations is confusing (on the index itself, or on the descriptors in the index, or on the image config itself).
For this reason it can vary the information that is wanting to be indexed.

As for the docker-registry, it is still up in the air how any of these annotations will even be exposed. I would venture to say they won't be at all. Historically community folks wanted to expose LABELS via the registry api and that never happened. But it would be very useful for projects like flatpak, because then their OCI images could re-use infrastructure like a docker registry.

[rhatdan]

@owtaylor @mtrmac @vbatts @vrothberg Is this fixed now? Can we close this issue?

[mtrmac]

I don’t really think of annotations in the (manifest) “index” as part of the image (which has the manifest as the top-level object). Unless the OCI registry standard deviates from docker/distribution enough to support this extra metadata (@vbatts ?), I don’t see how we could preserve them.

[owtaylor]

Flatpak doesn't care any more about annotations on the index (if it ever did.) But annotations on the manifest have to be preserved. I claimed that wasn't the case in the initial description, but I think it's likely working now since we do use skopeo for uploading Flatpaks from OSBS.

…

[mtrmac]

Flatpak doesn't care any more about annotations on the index (if it ever did.) But annotations on the manifest have to be preserved. I claimed that wasn't the case in the initial description, but I think it's likely working now since we do use skopeo for uploading Flatpaks from OSBS.

Conversions from OCI to schema2 do discard annotations AFAICS; so it probably works only if the registry can natively accept OCI manifests.

[owtaylor]

The manifest annotations issue was likely fixed by: containers/image#338 In general, I think this can be closed... though if/when pushing and pulling image indexes as image indexes is supported (#400) I would expect that the image indexes would be literally persevered, included annotations. (We don't need that for Flatpak, but it would be potentially problematical for other uses.)

[mtrmac]

#400 seems to be something quite irrelevant.

containers/image#400 OTOH, does involve somehow dealing with multi-arch OCI images, and that code does preserve the annotations I think. But that’s code, and functionality, that does not currently really exist, so keeping an issue filed against it does not quite make sense. So, closing.

Thanks!