Skip to content

Commit

Permalink
Force using /etc/containerd/certs.d for registry config.
Browse files Browse the repository at this point in the history
This is a breaking change, announced in release v0.20. See
https://kind.sigs.k8s.io/docs/user/local-registry/ how to setup a local
registry.

Note: users who used to patch the containerd config to set explicitly:

[plugins."io.containerd.grpc.v1.cri".registry]
  config_path = "/etc/containerd/certs.d"

should now remove this patch as it is now kind's default configuration.
  • Loading branch information
Romain-Geissler-1A committed May 3, 2024
1 parent 19df3db commit bb1973b
Show file tree
Hide file tree
Showing 4 changed files with 17 additions and 39 deletions.
3 changes: 3 additions & 0 deletions images/base/files/etc/containerd/config.toml
Original file line number Diff line number Diff line change
Expand Up @@ -40,3 +40,6 @@ version = 2
tolerate_missing_hugepages_controller = true
# restrict_oom_score_adj needs to be true when running inside UserNS (rootless)
restrict_oom_score_adj = false

[plugins."io.containerd.grpc.v1.cri".registry]
config_path = "/etc/containerd/certs.d"
10 changes: 1 addition & 9 deletions pkg/cluster/nodeutils/util_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -150,15 +150,7 @@ func TestParseSnapshotter(t *testing.T) {
key_model = "node"
[plugins."io.containerd.grpc.v1.cri".registry]
config_path = ""
[plugins."io.containerd.grpc.v1.cri".registry.auths]
[plugins."io.containerd.grpc.v1.cri".registry.configs]
[plugins."io.containerd.grpc.v1.cri".registry.headers]
[plugins."io.containerd.grpc.v1.cri".registry.mirrors]
config_path = "/etc/containerd/certs.d"
[plugins."io.containerd.grpc.v1.cri".x509_key_pair_streaming]
tls_cert_file = ""
Expand Down
20 changes: 10 additions & 10 deletions site/content/docs/user/private-registries.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,22 +91,22 @@ See Google's [upstream docs][keyFileAuthentication] on key file authentication f
#### Use a Certificate

If you have a registry authenticated with certificates, and both certificates and keys
reside on your host folder, it is possible to mount and use them into the `containerd` plugin
patching the default configuration, like in the example:
reside on your host folder, it is possible to mount to docker config which is compatible
with containerd, like in this example:

{{< codeFromInline lang="yaml" >}}
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
# This option mounts the host docker registry folder into
# the control-plane node, allowing containerd to access them.
# the control-plane node, allowing containerd to access them.
extraMounts:
- containerPath: /etc/docker/certs.d/registry.dev.example.com
hostPath: /etc/docker/certs.d/registry.dev.example.com
containerdConfigPatches:
- |-
[plugins."io.containerd.grpc.v1.cri".registry.configs."registry.dev.example.com".tls]
cert_file = "/etc/docker/certs.d/registry.dev.example.com/ba_client.cert"
key_file = "/etc/docker/certs.d/registry.dev.example.com/ba_client.key"
{{< /codeFromInline >}}
hostPath: /etc/containerd/certs.d/registry.dev.example.com
{{< /codeFromInline >}}

Note that if you have a hosts.toml file inside the registry configuration, this file needs
to explicitly mention the TLS certificates/keys, see the [CRI documentation][criDocumentation]

[criDocumentation]: https://github.com/containerd/containerd/blob/main/docs/hosts.md
23 changes: 3 additions & 20 deletions site/static/examples/kind-with-registry.sh
Original file line number Diff line number Diff line change
Expand Up @@ -10,24 +10,7 @@ if [ "$(docker inspect -f '{{.State.Running}}' "${reg_name}" 2>/dev/null || true
registry:2
fi

# 2. Create kind cluster with containerd registry config dir enabled
# TODO: kind will eventually enable this by default and this patch will
# be unnecessary.
#
# See:
# https://github.com/kubernetes-sigs/kind/issues/2875
# https://github.com/containerd/containerd/blob/main/docs/cri/config.md#registry-configuration
# See: https://github.com/containerd/containerd/blob/main/docs/hosts.md
cat <<EOF | kind create cluster --config=-
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
containerdConfigPatches:
- |-
[plugins."io.containerd.grpc.v1.cri".registry]
config_path = "/etc/containerd/certs.d"
EOF

# 3. Add the registry config to the nodes
# 2. Add the registry config to the nodes
#
# This is necessary because localhost resolves to loopback addresses that are
# network-namespace local.
Expand All @@ -43,13 +26,13 @@ for node in $(kind get nodes); do
EOF
done

# 4. Connect the registry to the cluster network if not already connected
# 3. Connect the registry to the cluster network if not already connected
# This allows kind to bootstrap the network but ensures they're on the same network
if [ "$(docker inspect -f='{{json .NetworkSettings.Networks.kind}}' "${reg_name}")" = 'null' ]; then
docker network connect "kind" "${reg_name}"
fi

# 5. Document the local registry
# 4. Document the local registry
# https://github.com/kubernetes/enhancements/tree/master/keps/sig-cluster-lifecycle/generic/1755-communicating-a-local-registry
cat <<EOF | kubectl apply -f -
apiVersion: v1
Expand Down

0 comments on commit bb1973b

Please sign in to comment.