Define --registry flag behaviour

[rjmateus]

For public cloud and internal registry usage, we must have a way to overwrite registry to where we want to pull the images from.
This card aims to agree on how to make it, and document what should be implemented.

For podman we always want to use the local RPM image if available. Local images are located at /usr/share/suse-docker-images/native.
The metadata file will have the image name and available tags. The name in the RPM image is independent from the registry (example: suse-manager-5.0-x86_64-server), and is computed from the image namespace defined on SCC.

example:
Image to use: registry.suse.com/suse/manager/5.0/x86_64/server
Will look at the file system by image name suse-manager-5.0-x86_64-server

Our goals are:

• Allow to set a different registry for cloud and private registry.
• Always look for the local image (RPM) before trying to download it as much as possible

Proposed Solution

Each image used in the deployment will have its configuration as we have now. Example: --image, --coco-image, --hubxmlrpc-image.
Those properties will be setting the image path/namespace, and can also set the tag name.

The registry flag will set the prefix. It can be just the registry FQDN, or also include some path prefix. Example:

• --registry=myreg.com
• --registry=myreg.com/prefix/path/

We always give priority to local images, but searching with the image path defined on image specific path flags. The values on those flags will be different on SUMA or Uyuni. The registry flag will affect only the online image pull.

If the user defines only the registry all images are affected with the same prefix.
If they want to change the default namespace, users should define it for each image one by one, with the correct location. Reason: if they start to play around with names, we cannot know how imaginative they can get, so define them individually.

Scenarios:

• Cloud registry: Only set the registry, should use the same image namespace and all should work with local lookup
• Private registry:
  • Keep same namespace: same as cloud, set registry flag and all should work with local lookup
  • Change namespace or tags: Define the registry (with or without prefix) and then for each image define the path.

Examples

Default for --registry: registry.suse.com
Default image path --image: suse/manager/5.0/x86_64/server
Default image tag --tag: 5-0

No flags defined

-- local look-up with
- ImageName: suse/manager/5.0/x86_64/server
- Tag: 5.0.0

• If fail, fallback to remote look-up with
  • Image: registry.suse.com/suse/manager/5.0/x86_64/server
  • Tag: 5.0.0

Set the Registry

registry=cloud.com

Look up locally with the default image path. Remote will use the defined registry + the image path

-- local look-up with
- ImageName: suse/manager/5.0/x86_64/server
- Tag: 5.0.0

• If fail, fallback to remote look-up with
  • Image: cloud.com/suse/manager/5.0/x86_64/server
  • Tag: 5.0.0

Flags set for tag and registry

registry=cloud.com
Tag: 5.0.0-tag

Look up locally with the default image path. Remote will use the defined registry + the image path

-- local look-up with
- Image: suse/manager/5.0/x86_64/server
- Tag: 5.0.0-tag

• If fail, fallback to remote look-up with
  • Image: cloud.com/suse/manager/5.0/x86_64/server
  • Tag: 5.0.0-tag

in this case tag will not be found, and we will move to a remote download

Flags set for image, tag and registry

registry=cloud.com
Image: my/path/server
Tag: 5.0.0-tag

Look-up locally with the passed image path and tag. Fall back to remote using the image passed, using the passed registry

-- local look-up with
- Image: my/path/server
- Tag: 5.0.0-tag

• If fail, fallback to remote look-up with
  • Image: cloud.com/my/path/server
  • Tag: 5.0.0-tag

In this case, it will never find the image name, since it's not our image path. Will use remote download.

#Related links:

• Fix --registry option (bsc#1226436) #367
• https://bugzilla.suse.com/show_bug.cgi?id=1226436
• pull image from a private registry and fix pull policy #332

[aaannz]

I think there are some misunderstandings what is generally used and what are rules we/suse added on ourselves.

eg.

For podman we always want to use the local RPM image if available. Local images are located at /usr/share/suse-docker-images/native.
The metadata file will have the image name and available tags. The name is independent from the registry in use (example: suse-manager-5.0-x86_64-server), and is computed from the image namespace.

That example right there is not correct. Our images are NOT independent of our registry. Our registry, we, added this namespace to the images. When you do

skopeo copy --tls-verify=false docker://registry.suse.de/devel/galaxy/manager/head/containerfile/suse/manager/5.0/x86_64/server:5.0.0 docker://localhost:5000/server

to get local copy of the image, you will not have this path anywhere:

# podman search --tls-verify=false localhost:5000/
NAME                         DESCRIPTION
localhost:5000/server

I argue that that namespace, that ugly string we/obs added, is part of our infrastructure and part of our registry name because it's not consistent across different registries.

Do we know for certain that cloud registries will use the very same path like we do?

[rjmateus]

@aaannz when I mentioned the image is independent from the registry I was mentioning the image delivered as RPM, where the image name on the metadata file doesn't contain the registry where the image is published. It only contained a concatenation of the image path.

One main goal for me is being able to have good sane defaults and avoid the need for users to enter too many fields. That is why I was thinking if users keep the same path in the internal registry, then they could add just the registry, otherwise they would need to specify the images one by one, since we cannot know what they defined as name.

Skopeo can also be used to sync a full tree, and can re-use the same path. Can you try skopeo sync --tls-verify=false --src docker --dest docker registry.suse.de/devel/galaxy/manager/head/containerfile/suse/manager/5.0/x86_64/server localhost:5000
It should copy all tags with the same path

Do you have any suggestions on how would be the best approach on this one?

For the cloud I need to double check, but as far as I know they will keep the same structure as we have on registry.suse.com.

Thank you

[aaannz]

I did not notice sync command for the skopeo, however you need to add --scoped option to sync correctly complete prefix path.

Do you have any suggestions on how would be the best approach on this one?

If we assume that the name of the SUMA image will always be suse/manager/5.0/x86_64/$image_name, i.e. we will prepend this prefix to any image name then:

1. switch between registry.suse.com and registry.cloud will work by using plain --registry option.
2. switch between registry.suse.com to registry.suse.de will require --registry=registry.suse.de/suse/sle-15-sp6/update/products/manager50/containerfile, which I agree with.
3. to help internal testing, we may add to sumaform or other docs an information about custom registries.conf.d mapping, e.g.
   "suse/suma_devel" = "registry.suse.de/.../containerfile"
4. specifying different images in tools can use their short names without any prefix

[rjmateus]

The --skoped will also add the registry fqdn, not just the path, IIRC.

I think I lost you :)

Let me check if I understand you. 1th we need to check with public cloud if the image paths will be the same also in the cloud.

The --registry will be the prefix to be used. In the cloud, users can just specify the fqdn for the registry. For internal registry if they use the exact same path, is the same (or if then add an extra prefix, like with the scoped flag of skopeo sync).

By default, registry will be registry.suse.com (different for uyuni) and it will be joined with suse/manager/5.0/x86_64/$image_name for each image.

What if users specify the registry and for example --image. Should we use or ignore the registry flag? I would say we ignore the registry flag, since you are changing the image user could just set the full path.

[aaannz]

The --skoped will also add the registry fqdn, not just the path, IIRC.

Indeed it did. Without --scoped it does not copy the path. With --scoped it prepends registry fqdn.
But I checked the docs and in the docs we instruct to manually use correct path. So I guess this is fine.

I think I lost you :)

Let me check if I understand you. 1th we need to check with public cloud if the image paths will be the same also in the cloud.

Yes

The --registry will be the prefix to be used. In the cloud, users can just specify the fqdn for the registry. For internal registry if they use the exact same path, is the same (or if then add an extra prefix, like with the scoped flag of skopeo sync).

By default, registry will be registry.suse.com (different for uyuni) and it will be joined with suse/manager/5.0/x86_64/$image_name for each image.

Yes, assuming cloud uses that path as well

What if users specify the registry and for example --image. Should we use or ignore the registry flag? I would say we ignore the registry flag, since you are changing the image user could just set the full path.

I think we should not surprise our users:
--image URL --registry FQDN -> if used in one command do FQDN/URL, maybe check if image is indeed available and fail fast?
--image URL -> pass URL without modifications and let podman/k8s deal with what registry to use based on system config
--registry FQDN or --registry FQDN/PATH -> take our image names (which contains suse/manager/5.0/x86_64 prefix, assuming cloud uses it as well) and prepend it with value passed as registry.

[rjmateus]

Initially for cloud only images in RPM will be available. In that case the RPM is the same as from SCC, and we can trust the path will be the same.

[rjmateus]

--registry FQDN or --registry FQDN/PATH -> take our image names (which contains suse/manager/5.0/x86_64 prefix, assuming cloud uses it as well) and prepend it with value passed as registry.

I didn't got this one, you have registry two time

[aaannz]

--registry FQDN or --registry FQDN/PATH -> take our image names (which contains suse/manager/5.0/x86_64 prefix, assuming cloud uses it as well) and prepend it with value passed as registry.

I didn't got this one, you have registry two time

Yes, those are two examples. One where registry is just FQDN, another example to demonstrate that part of the registry can be also some path and not plain fqdn. For example to use with internal registry.

[juliogonzalez]

Sorry to be picky here but: shouldn't --registry be used only to point to the registry, meaning the hostname (FQDN) (and maybe port for the registry?

That's what I'd expect as a user at least.

Don't we have --image for the path?

[aaannz]

Sorry to be picky here but: shouldn't --registry be used only to point to the registry, meaning the hostname (FQDN) (and maybe port for the registry?

That's what I'd expect as a user at least.

Don't we have --image for the path?

What we want is to be able to switch all images from one registry to another.
If you have registries using different namespaces, e.g. registry.suse.com and registry.suse.de when you will insist on using only FQDN, then you will need either add extra option registry-path or namespace; or write --image for all images you want. For some it is currently only server, but for some it's coco image, hub image, migration image and in the future it will be much more.

Nowhere it is defined that registry must be only fqdn.

Btw. with agreed approach if user uses --registry fqdn --image full_path/image it will work and it will work the same as --registry fqdn/namespace --image image

[cbosdo]

We already start to see confusion where --registry fqdn/path/to/uyuni computes images like fqdn/path/to/uyuni/uyuni. This is not consistent: either the path can be in the registry but then it should be the full path or it shouldn't be in the registry and it will be a pain for us since OBS images for Master will have to be duplicated for each --*-image parameter.

[rjmateus]

We may be missing better documentation on the tool, but the behaviour is clear, and the current implementation will cope with several scenarios: Cloud Registry, copy images and keep the SUSE/UYUNI published path, with or without a prefix.
If uses change the path compitelly, we cannot assume that they will keep the last part of the default path.

The issue we have seen was a miss configuration on sumaform that was not updated for server after the changes we made to the registry flag.