Error getting hosted zone name ... for ClusterIssuer without hostedZoneName defined.

[hetii]

Hello dear.
I use such ClusterIssuer.
Note here that clientID is set to: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
and hostedZoneName is not defined:

apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
  name: letsencrypt-production
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: xxx@xxx.com
    privateKeySecretRef:
      name: letsencrypt
    solvers:
    - dns01:
        azuredns:
          # Service principal client id
          clientID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
          tenantID: ...
          subscriptionID: ...
          # Secret with the password
          clientSecretSecretRef:
            key: password
            name: secret-azuredns-config
          # Name of the DNS Zone
          # https://github.com/jetstack/cert-manager/issues/1459
          # hostedZoneName: example.com
          # Resource group where the DNS Zone is located
          resourceGroupName: aks-cluster-dev

The clientID "xxxxxxxx..." have service principals created by terraform and access only for exactly given set of domains, let say (example.com, foo.com, bar.com etc).

When I apply it I got errors like:

Error getting hosted zone name for: 
 _acme-challenge.example.com., Zone com. not found in AzureDNS for domain
 _acme-challenge.example.com.. Err: dns.ZonesClient#Get: 
Failure responding to request: StatusCode=403 -- Original Error: autorest/azure:
Service returned an error. Status=403 Code="AuthorizationFailed"
Message="The client 'bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb' 
with object id 'bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb' 
does not have authorization to perform action
 'Microsoft.Network/dnsZones/read' over scope 
'/subscriptions/<my subscription here>/resourceGroups/aks-cluster-dev/providers/Microsoft.Network/dnsZones/com' 
or the scope is invalid.

As we can see there is two issue, first no clue why errors talk about client "bbbbbbbb" as I use "xxxxxxxx" and when change clientID to let say fake "yyyyyy" then got proper message, that such principal don't exist in my azure tenatID.

The second issue is that without hostedZoneName defined in ClusterIssuer, the cert-manager trying to read zones one level up, so in this case starting from .com ("Microsoft.Network/dnsZones/com'") and of course raise above errors as I don't manage primary zones :)

On other hand when I set hostedZoneName and clientID back to "xxxxxxxx..." then I see proper acme challenge that are set in my zone.

So lets say that now that I need to manage 100 different zones, do I need to create then 100 ClusterIssuers for them with only different hostedZoneName?

cert-manager version: v0.15.0-alpha.0
Kubernetes: v1.16.7
Best Regards.

[wallrj]

@hetii I don't know why the client-id should be mis-reported in the errors.
The empty hostedZone bug #1459 (reported by @tommy-dk) has been fixed in #1466 by @logicfox since March 2019.

Perhaps one of them can comment on whether this is the same issue.

[wallrj]

/help

[jetstack-bot]

@wallrj:
This request has been marked as needing help from a contributor.

Please ensure the request meets the requirements listed here.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

Details

In response to this:

/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[wallrj]

/priority backlog

[yves-vogl]

Maybe there's another issue you are not aware of.
Lets image you have 3 zones in a resource group.

root.example.com
a.root.example.com
b.root.example.com

As "a" and "b" are delegated from root.example.com they should be used for certificates e.g. (host.b.root.example.com in b.root.example.com). When not specifying the hostZoneName the cert-manager chooses root.example.com to set b.root.example.com TXT record which leads to an unresolvable record.

When specifying more than one hostedZoneName in a single issuer (e.g. a.root.example.com and b.root.example.com) and it's not able to figure out the correct zone.

[rneto12]

Maybe there's another issue you are not aware of.
Lets image you have 3 zones in a resource group.

root.example.com
a.root.example.com
b.root.example.com

As "a" and "b" are delegated from root.example.com they should be used for certificates e.g. (host.b.root.example.com in b.root.example.com). When not specifying the hostZoneName the cert-manager chooses root.example.com to set b.root.example.com TXT record which leads to an unresolvable record.

When specifying more than one hostedZoneName in a single issuer (e.g. a.root.example.com and b.root.example.com) and it's not able to figure out the correct zone.

I need to use rfc2136 in a scenario like described (on my bare-metal kubernetes cluster and bare-metal bind server), but the ClusterIssuer only try to update the root zone, according to the bind logs.
Unfortunatelly there's no hostedZoneName option in rfc2136 to force delegated zone/domain (as in https://cert-manager.io/docs/configuration/acme/dns01/#delegated-domains-for-dns01).
I'm using this config:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    email: mail@domain.com
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    preferredChain: "ISRG Root X1"
    privateKeySecretRef:
      name: letsencript-staging
    solvers:
    - dns01:
        rfc2136:
          nameserver: ns.domain.com:53
          tsigKeyName: sub.domain.com.
          tsigAlgorithm: HMACSHA512
          tsigSecretSecretRef:
            name: tsig-secret
            key: tsig-secret-key

bind log:

Mar  2 20:30:53 ns named[4867]: client kube-server-IP#45589/key sub.domain.com: update 'domain.com/IN' denied

I already tested tsig name and key using nsupdate for the sub.domain.com, and it works.

When testing directly domain.com in clusterissuer config (unsing the corresponding tsig name and key) it also works, but I have to use a subdomain.

I think my issue its similar to this, but let me know if I need open new issue specific to rfc2136.

Version
Kubernetes 1.20.2
Cert-Manager 1.2 (helm)

[hetii]

This issue back to me almost one year the time when I create it, and here is the case.
Usually I create a 3 separate kubernetes clusters, each has own cert-manager and each manage his own given domain,
so it looks like below:

cluster:   domain:
prod       example.com
test       test.example.com
dev        dev.example.com  

And now the tricky part, if dev/test cluster is created first and zone example.com doesn't exists in azure cert-manager based on this function
return empty hostedZoneName and correctly manage test.example.com and/or dev.example.com.

Now if prod cluster and his example.com domain is created before dev/test, then cert-manager from dev/test
detect that top level exist and try to use it instead of subdomain, and as I don't give it permission from dev/test to manage zone from prod, it fails.

The odd think regarding checking top domain is in that we have resourceGroupName arugment for Azure ClusterIssuer and this group is different for dev/test/prod environments, so this is next subject to check why this group is not taken into the game when top level domain is checked.

So I think that above function should check if given instance of cert-manager see top level domain and has rights to manage it, otherwise if not then subdomain should be manage.

[jetstack-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Send feedback to jetstack.
/lifecycle stale

[jetstack-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Send feedback to jetstack.
/lifecycle rotten
/remove-lifecycle stale