ClusterIssuer w/ Cloudflare DNS01 cannot find Secret

[mikejk8s]

Is this a BUG REPORT or FEATURE REQUEST?:

Uncomment only one, leave it on its own line:

/kind bug

/kind feature

What happened:
I deployed via helm today and spent quite a bit of time trying to get ClusterIssuer working with Cloudflare DNS01 validation. Everything is being deployed to the default namespace. I'm going to ignore the portions installed by Helm as all I've done beyond that is add my API key, ClusterIssuer and Certificate.

When I try to generate a certificate using ClusterIssuer I get Secret Not Found errors;

[cert-manager-solid-echidna-57dcd6d5fd-klpcx cert-manager] I0117 22:31:19.037700       1 sync.go:242] Error preparing issuer for certificate: error presenting acme authorization for domain "acme02.mydomain.com": error getting cloudflare service account: secret "cloudflare" not found

When I convert my Certificate and ClusterIssuer to be an Issuer only (remove kind:ClusterIssuer from cert and switch kind to Issuer in Issuer.yaml) the Issuer can find the secret fine.

The only change at all is switching from ClusterIssuer to Issuer.

cert-manager-solid-echidna-57dcd6d5fd-klpcx cert-manager] I0117 22:33:55.045866       1 controller.go:187] certificates controller: syncing item 'default/acme02-com'
[cert-manager-solid-echidna-57dcd6d5fd-klpcx cert-manager] I0117 22:33:55.045931       1 sync.go:107] Error checking existing TLS certificate: secret "acme02-com-tls" not found
[cert-manager-solid-echidna-57dcd6d5fd-klpcx cert-manager] I0117 22:33:55.045989       1 sync.go:238] Preparing certificate with issuer
[cert-manager-solid-echidna-57dcd6d5fd-klpcx cert-manager] I0117 22:33:55.046459       1 prepare.go:239] Compare "" with "https://acme-staging.api.letsencrypt.org/acme/reg/5404187"
[cert-manager-solid-echidna-57dcd6d5fd-klpcx cert-manager] I0117 22:35:00.643613       1 sync.go:248] Issuing certificate...
[cert-manager-solid-echidna-57dcd6d5fd-klpcx cert-manager] I0117 22:35:01.378389       1 helpers.go:165] Setting lastTransitionTime for Certificate "acme02-help-com" condition "Ready" to 2018-01-17 22:35:01.378375063 +0000 UTC m=+6196.269145686
[cert-manager-solid-echidna-57dcd6d5fd-klpcx cert-manager] I0117 22:35:01.398256       1 sync.go:269] Certificated issued successfully

What you expected to happen:

ClusterIssuer should be able to reach Secret in the same namespace. Or let me know what I'm doing wrong, thanks!

How to reproduce it (as minimally and precisely as possible):

I've tried this multiple times before submitting ticket

apiVersion: certmanager.k8s.io/v1alpha1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
  namespace: default
spec:
  acme:
    server: https://acme-staging.api.letsencrypt.org/directory
    email: me@me.com
    privateKeySecretRef:
      name: letsencrypt-staging
    dns01:
      providers:
      - name: cloudflare
        cloudflare:
          email: me@me.com
          apiKeySecretRef:
            name: cloudflare
            key: api

apiVersion: certmanager.k8s.io/v1alpha1
kind: Certificate
metadata:
  name: acme03-com
spec:
  secretName: acme03-com-tls
  issuerRef:
    name: letsencrypt-staging
    kind: ClusterIssuer
  commonName: acme03.me.com
  dnsNames:
  - acme03.me.com
  acme:
    config:
    - dns01:
        provider: cloudflare
      domains:
      - acme03.me.com

apiVersion: v1
kind: Secret
metadata:
  name: cloudflare
  namespace: default
data:
   api: base64key

Anything else we need to know?:

Environment:

• Kubernetes version (use kubectl version): 1.86.0-GKE
• Cloud provider or hardware configuration**: GKE
• Install tools: Helm
• Others:

[munnerz]

The default namespace for these cluster scoped supporting resources is kube-system. If you've not provided a custom --cluster-resource-namespace, you should make sure your cloudflare secret is in the kube-system namespace 😄.

[mikejk8s]

I figured I was missing something, I thought the cluster-issuer was what needed direct access to the key so it was confusing me.

Thanks a lot!

[sheerun]

@mikejk8s I think this issue should be opened because and either ClusterIssuer should be disallowed to be created in namespaces other than kube-system, or Certificate should read secrets from the same namespace it is in. Otherwise it creates much confusion.

[ndarilek]

I'm seeing this failure case here, and am fairly new to k8s. How would I go about modifying the above config to ensure that the secret is created in the correct namespace? As of now, kubectl get secrets --all-namespaces isn't showing me a secret named per the issuer config, so I don't know if I'm hitting this issue or something else.

DO I add a namespace to the ClusterIssuer? Do I add a namespace to the DNS01 configuration like so?:

        cloudflare:
          email: me@example.com
          apiKeySecretRef:
            name: cloudflare-api-key-secret
            namespace: kube-system
            key: NO

Or something else entirely? Thanks for the help.

[pentago]

For everyone who arrives here looking for a solution to this particular problem, the above comment is right, Cloudflare API secret has to reside in kube-system namespace or namespace where cert-manager is running in. As per official docs:

When referencing a Secret resource in ClusterIssuer resources (eg apiKeySecretRef) the Secret needs to be in the same namespace as the cert-manager controller pod.

Just found this out the hard way 👎 .

[colvint]

Moved the secret to kube-system. Still didn't work. Re-read the error (and comments above):

or namespace where cert-manager is running in

Put the secret in cert-manager namespace was the ticket. Slightly dismaying for those of us using sealed secrets. Means we need to helm install sealed secrets into cert-manager namespace as well.