[QUESTION/HELP] Unable to use private Helm registry

[brsolomon-deloitte]

I would like to deploy metrics-server from registry1.dso.mil to k3d.

I am able to run this successfully:

helm pull oci://registry1.dso.mil/bigbang/metrics-server --version 3.10.0-bb.0

However, when I try to install this using helm.cattle.io/v1 HelmChart, I get a 401 Unauthorized error.

To reproduce:

cat << EOF > /tmp/k3d-config.yaml
apiVersion: k3d.io/v1alpha5
kind: Simple
metadata:
  name: mycluster
servers: 1
agents: 2
options:
  k3d:
    disableLoadbalancer: true
  k3s:
    extraArgs:
      - arg: "--disable=traefik"
        nodeFilters:
          - server:*
      - arg: "--disable=metrics-server"
        nodeFilters:
          - server:*
EOF

cat << EOF > /tmp/k3d-registry-config.yaml
mirrors:
  registry1.dso.mil:
    endpoint:
      - oci://registry1.dso.mil
configs:
  registry1.dso.mil:
    auth:
      username: <REDACTED>
      password: <REDACTED>
EOF

k3d cluster create --config /tmp/k3d-config.yaml --registry-config /tmp/k3d-registry-config.yaml

kubectl apply -f - <<EOF
apiVersion: helm.cattle.io/v1
kind: HelmChart
metadata:
  name: metrics-server
  namespace: kube-system
spec:
  chart: oci://registry1.dso.mil/bigbang/metrics-server
  version: 3.10.0-bb.0
  targetNamespace: kube-system
EOF

Then:

kubectl logs -n kube-system job/helm-install-metrics-server

Output:

+ echo 'Installing helm_v3 chart'
+ helm_v3 install --namespace kube-system --version 3.10.0-bb.0 metrics-server oci://registry1.dso.mil/bigbang/metrics-server
Error: INSTALLATION FAILED: pulling from host registry1.dso.mil failed with status code [manifests 3.10.0-bb.0]: 401 Unauthorized

What do I have wrong here?

I am also able to do this successfully:

$ helm registry login registry1.dso.mil
Username: <REDACTED>
Password: <REDACTED>
Login Succeeded

[iwilltry42]

Hey 👋
The registry config is for containerd running in the K3s nodes, so it won't be used by pods (like the Helm Chart controller).
Also I guess it wouldn't work for Helm in general 🤔

[iwilltry42]

The HelmChart CRD seems to have fields for Auth though: https://github.com/k3s-io/helm-controller/blob/master/pkg/apis/helm.cattle.io/v1/types.go#L37-L40