Insecure private registry

[maitredede]

Describe the problem to be solved

I have a private local registry (http only) that I have configured with a hosts.toml file. When starting spegel, this configuration is removed.

/etc/containerd/certs.d/myhost.local/hosts.toml :

server = "http://myhost.local"

[host."http://myhost.local:80"]
capabilities = ["pull", "resolve", "push"]
skip_verify = true

Proposed solution to the problem

• One solution would be to add a way to skip domains so folder is not handled --skip-domain=myhost.local

• One other solution would be to have a way to set some registry options and generate corresponding configuration. For example `--registry "http://myhost.local:80"=skip_verify=true"

[phillebaba]

The Helm chart already has the field spegel.appendMirrors that will cause Spegel to append the mirror configuration after any existing configuration. Could you check if this solves your problems?

https://github.com/spegel-org/spegel/tree/main/charts/spegel

[maitredede]

This fields adds each entry to each generated file, but I have one registry mirror per mirrored registry. With the spegel.appendMirrors field, when trying to download one image, it will ask all mirrors before getting the right one.

It will work in almost cases, but it generates lots of 404 for no purposes, and how does it work if two registries has the same image tag ? (only the hosting registry that is different).

That is why I would prefer to have a finer configuration. And also a way to handle insecure registries.

[phillebaba]

Ok so you want to keep the original configuration, only for this specific registry? Why would you not want to use Spegel for the registry anyways?

I am having trouble thinking of how this would be implemented, maybe like a "persist backup" setting? Eventually very specific configuration needs fall out of scope for Spegel and you are better off disabling it in Spegel and creating your own init container to do it.

[lindeskar]

Looks like the Helm value is renamed to prependExisting?

Something for the docs; it seems to have no effect if no registries are defined in mirroredRegistries (i.e. all registries are mirrored). I see all config under /etc/containerd/certs.d/ is replaced even with prependExisting: true.

[David2011Hernandez]

I have TLS certs for my custom registry also stored in /etc/containerd/certs.d but spegel pods removes all the contents and writes its configuration, even with prependExisting: true

[phillebaba]

@David2011Hernandez are you able to store the certs anywhere else other than the certs.d directory?

[David2011Hernandez]

I could if then containerd will pick them when pulling images from my registries. I just do not know another way to tell containerd to use certain secrets for certain hosts otherwise

I have full control of this cause I do it via a Daemonset, in case there is other way I could try to do it

[phillebaba]

Could you share how your configuration file looks like?

[David2011Hernandez]

---
grafanaDashboard:
  enabled: true

spegel:
  logLevel: "INFO"
  mirroredRegistries: []
  # here i have tried to add my custom registries but due to certs overwrite it does not work
  # what i mean here is that spegel correctly sets my registries but then it fails on imagePull
  additionalMirrorTargets: []
  # same here, spegel overwrites the whole directory 
  # Here again spegel correctly sets the mirror target in the config
  prependExisting: true

I have a hacky way to make it work, given that I control the daemonset that copies my certificates, using an init container I wait for spegel pod to be in node before I copy my certs leaving spegel modification intact

[phillebaba]

This should be fixed in #1067.