DNS validation fails but it's wrong

[r00ta]

I've tried to install a cluster with

with the following dns settings

but the validations fails due to the dns wildcard certificate (that does not exists at all!)

Is that a known issue?

[omertuc]

/close User figured out it's some weird router behavior. Created #4482 to give users more information in this situation

[r00ta]

Thank you so much @omertuc for the support!

[jklare]

Just in case anybody else stumbles across this thread:

1. Congratulations that you made it this far!
2. I encountered a very similar issue and could solve it by reconfiguring my DHCP server to stop announcing the apps.<clustername>.<basedomain> as option domain-search (dhcpd.conf)
3. The issue here seems to be, that the validator, as well as the coredns running on the fully configured cluster lateron, will respect the domain-search option announced by the DHCP server for the network and therefore append those to any DNS requests in case the initial one returns NXDOMAIN. This means, that the first request of the validator will go towards the local DNS resolver on the cluster node itself and this one will respect the domain-search parameters announced by the DHCP server earlier. So the local resolver will first ask the upstream DNS for validateNoWildcardDNS.<clustername>.<basedomain> and get the expected NXDOMAIN. It will then append the domain-search (e.g. apps..) and ask for validateNoWildcardDNS.<clustername>.<basedomain>.apps.<clustername>.<basedomain> and get back the IP address configured for the wildcard A record of *.apps.<clustername>.<basedomain>.

[omertuc]

Seems like an issue in how our validation works. Thanks for the explanation / looking into it. I will open an internal ticket to see how we can address it

[jklare]

The strange thing for me here is, that although i agree and the validator should probably output a different error message, it is still somehow correct when saying "the cluster will not work with the current DNS/DHCP configuration". I initially ran into this issue since i saw the core DNS on my running cluster resolving service names (e.g. myservice.mynamespace.svc.cluster.local) to the address of my wildcard record (for *.apps.<clustername>.<basedomain>). While debugging i found out, that core DNS was also using the underlying local resolver and instead of resolving mservice.mynamespace.svc.cluster.local, it was appending the aformentioned domain-search parameter announced by my external dhcpd and was actually getting an answer for mservice.mynamespace.svc.cluster.local.apps.<clustername>.<basedomain>, which was obviously wrong. I am still investigating how to fix the full issue...

[omertuc]

Good to know that because our proposed solution on internal Slack was to append . at the end to signal the resolver that the domain ends where we say it ends, no further appending should be done by the resolver.

But if the presence of your DHCP config breaks OCP itself... maybe we shouldn't do that. Perhaps we should first wait for OCP to be tolerant of such DHCP config and just improve the error message until then

[jklare]

Yeah, i think improving the error message would be a good first step. I will keep you posted once i figure out how to configure oc for this scenario :).