Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

unbound cannot resolve a hostname from a dedicated location #694

Closed
toralf opened this issue Jun 9, 2022 · 4 comments
Closed

unbound cannot resolve a hostname from a dedicated location #694

toralf opened this issue Jun 9, 2022 · 4 comments

Comments

@toralf
Copy link

toralf commented Jun 9, 2022

From a dedicated server (hosted by Hetzner in FI) I cannot resolve wdl1.pcfg.cache.wpscdn.com with unbound whereas it works fine here from Hamburg (DE) - from my desktop.

I tested at both systems with unbound version 1.13.2. and at the server with 1.15.0 too but w/o success.
The OS at both systems is a hardened stable Gentoo.

https://dnsviz.net/d/wdl1.pcfg.cache.wpscdn.com/dnssec/ shows a lot of warnings - but I do still wonder about the different behaviour.

Using 8.8.8.8 as a resolver works fine so far.
@toralf
Copy link
Author

toralf commented Jun 10, 2022

with the help of an IRC user U could narrow it down to

~ $ dig +trace ns7.jdgslb.com
NS i.root-servers.net. from server ::1 in 0 ms.
NS c.root-servers.net. from server ::1 in 0 ms.
NS g.root-servers.net. from server ::1 in 0 ms.
NS b.root-servers.net. from server ::1 in 0 ms.
NS k.root-servers.net. from server ::1 in 0 ms.
NS f.root-servers.net. from server ::1 in 0 ms.
NS a.root-servers.net. from server ::1 in 0 ms.
NS h.root-servers.net. from server ::1 in 0 ms.
NS j.root-servers.net. from server ::1 in 0 ms.
NS e.root-servers.net. from server ::1 in 0 ms.
NS l.root-servers.net. from server ::1 in 0 ms.
NS d.root-servers.net. from server ::1 in 0 ms.
NS m.root-servers.net. from server ::1 in 0 ms.
RRSIG NS 8 0 518400 20220623050000 20220610040000 47671 . mGixyZPwQCUGyvixqxejiY+5flCKYRWS3jBZR0H00Bjw83UFMzRUAmYY ECdviXVqNDDQ3BY9cHDrIRY0G2yHwy/GQ7Al+5MS3F7fLaLzHGkNYf10 Pntq0Siek2P9yA6AOo3KSXk0aqzCv+2mmQaCN/7eJ4V0r26H+y4+3iTk J4nMDjoZiJRMnUwLu0e/yQToKBJ79tPztV/5pd1wtwnbshQnrF1lA2yC JPRTNX2Ue4Sottk77S2lHoZVSIkSpAWHCIaUMcWxQYGsHvXnBG7qyJ4a EVH373JPH1LvUSasJQ97rwcmnPFNIq1T9pW3tfnLndi1CzJmEGLd/5Jy HuB1Ow== from server ::1 in 0 ms.
couldn't get address for 'ns3.jdgslb.com': failure
couldn't get address for 'ns4.jdgslb.com': failure
dig: couldn't get address for 'ns3.jdgslb.com': no more

at 65.21.94.49, whereas the same query works fine at 77.0.163.53 :

$ dig +trace ns7.jdgslb.com 

; <<>> DiG 9.16.27 <<>> +trace ns7.jdgslb.com
;; global options: +cmd
.                       76064   IN      NS      l.root-servers.net.
.                       76064   IN      NS      m.root-servers.net.
.                       76064   IN      NS      a.root-servers.net.
.                       76064   IN      NS      b.root-servers.net.
.                       76064   IN      NS      c.root-servers.net.
.                       76064   IN      NS      d.root-servers.net.
.                       76064   IN      NS      e.root-servers.net.
.                       76064   IN      NS      f.root-servers.net.
.                       76064   IN      NS      g.root-servers.net.
.                       76064   IN      NS      h.root-servers.net.
.                       76064   IN      NS      i.root-servers.net.
.                       76064   IN      NS      j.root-servers.net.
.                       76064   IN      NS      k.root-servers.net.
.                       76064   IN      RRSIG   NS 8 0 518400 20220623050000 20220610040000 47671 . mGixyZPwQCUGyvixqxejiY+5flCKYRWS3jBZR0H00Bjw83UFMzRUAmYY ECdviXVqNDDQ3BY9cHDrIRY0G2yHwy/GQ7Al+5MS3F7fLaLzHGkNYf10 Pntq0Siek2P9yA6AOo3KSXk0aqzCv+2mmQaCN/7eJ4V0r26H+y4+3iTk J4nMDjoZiJRMnUwLu0e/yQToKBJ79tPztV/5pd1wtwnbshQnrF1lA2yC JPRTNX2Ue4Sottk77S2lHoZVSIkSpAWHCIaUMcWxQYGsHvXnBG7qyJ4a EVH373JPH1LvUSasJQ97rwcmnPFNIq1T9pW3tfnLndi1CzJmEGLd/5Jy HuB1Ow==
;; Received 1097 bytes from 127.0.0.1#53(127.0.0.1) in 1 ms

com.                    172800  IN      NS      d.gtld-servers.net.
com.                    172800  IN      NS      j.gtld-servers.net.
com.                    172800  IN      NS      e.gtld-servers.net.
com.                    172800  IN      NS      b.gtld-servers.net.
com.                    172800  IN      NS      a.gtld-servers.net.
com.                    172800  IN      NS      f.gtld-servers.net.
com.                    172800  IN      NS      h.gtld-servers.net.
com.                    172800  IN      NS      c.gtld-servers.net.
com.                    172800  IN      NS      g.gtld-servers.net.
com.                    172800  IN      NS      i.gtld-servers.net.
com.                    172800  IN      NS      m.gtld-servers.net.
com.                    172800  IN      NS      l.gtld-servers.net.
com.                    172800  IN      NS      k.gtld-servers.net.
com.                    86400   IN      DS      30909 8 2 E2D3C916F6DEEAC73294E8268FB5885044A833FC5459588F4A9184CF C41A5766
com.                    86400   IN      RRSIG   DS 8 1 86400 20220623170000 20220610160000 47671 . dw9TVASqR8YlOdk9oXQr5RXk4/2SabmYnK/eKAvlp0buNxNlSpSCpBNO 4sillQQpS/DPiEmdLmOc1SbrAK9l7geww9o3eDDPqedIAS7xYmrVzdxI DlliZXvrwvZeM/vVn1PmdH8cuPPHy7U/wXICcPp+dTtDev5UgeCtB7nJ U7Jl2uRB3DBYgC3+Piv738PaN5Ta1P9W8u1oLaeKfgsS0aTcmZt4y/gD xacSeNfRRoqBtWhKTmeLujNKbMJUM5OOV2ZLXpfCdV/T95WazIszHua9 5DcPd7vIEm9HFWZh5NGa5V0K3zrQawl/VFD6khbxjZy6E1ipY1fxd5b4 usGmIg==
;; Received 1205 bytes from 192.36.148.17#53(i.root-servers.net) in 15 ms

jdgslb.com.             172800  IN      NS      ns3.jdgslb.com.
jdgslb.com.             172800  IN      NS      ns4.jdgslb.com.
CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN NSEC3 1 1 0 - CK0Q2D6NI4I7EQH8NA30NS61O48UL8G5 NS SOA RRSIG DNSKEY NSEC3PARAM
CK0POJMG874LJREF7EFN8430QVIT8BSM.com. 86400 IN RRSIG NSEC3 8 2 86400 20220616042421 20220609031421 37269 com. KyvXqZlnlDD84rmrofbdsDxX6HpGm1lcDjq5DgOom47xFRXU9CCTXrvr zXp5YQ4TKQzWBRWGwgB5GJc3Qtt4aknBzo7oNXGdQfu1Hy9O5OJqQJtK zgoerDZdkm574FmZJQuI0h1bylwBCUx53O/BJqtPG1mRvBq+Mwyr37tt gpw1h8SYesykMROfytEFh0BpkJ5v006+lxb9lGFsuFxxbA==
CM1DOLD580DB1KILKANSHS3H24KDO9LE.com. 86400 IN NSEC3 1 1 0 - CM1E0UL21HV06AQQPH5A7K89KNJG6HUK NS DS RRSIG
CM1DOLD580DB1KILKANSHS3H24KDO9LE.com. 86400 IN RRSIG NSEC3 8 2 86400 20220617051800 20220610040800 37269 com. IRZnCvj5Gy+7nn3zJ5RStRpmEi1SHXNnhp4AbmCj1OYUs/Ck8jtVZdYI DzJZPf7u0Pbx+RG4++dj9D5CvO+ucKo1mpttMFNrON2xo/jdsy/+SI09 ZMBF3FjW0Ni+cA8mlbGTHRvlfSqKv6VIeGNLj8wzKM12g6khlkXUvLBW M/leg1EPbp+j4KjgikcsR3ZMZIPqAxWFFFEqYh/KRmb1UA==
;; Received 812 bytes from 192.33.14.30#53(b.gtld-servers.net) in 16 ms

ns7.jdgslb.com.         600     IN      A       183.131.197.220
ns7.jdgslb.com.         600     IN      A       114.67.160.152
ns7.jdgslb.com.         600     IN      A       114.67.121.112
ns7.jdgslb.com.         600     IN      A       101.124.23.10
;; Received 107 bytes from 183.131.197.216#53(ns3.jdgslb.com) in 230 ms

@wcawijngaards
Copy link
Member

Yes the domain shows a number of warnings. I can resolve it here just fine, but I also see that it returns NXDOMAIN for intermediate labels, which it should not do. But unbound retries with the full query name, and thus recovers from the failure, and continue with query-minimisation to resolve. The server for jcloud-cdn.com. with name ns7.jdgslb.com. and ip 114.67.121.112 shows failures. I mean it has a number of timeouts, maybe one in ten queries. Also that does not stop a resolve, because of retries.

The link you have shows that another error exists, qiniudns.com has records next to a CNAME. That is likely to confuse the resolver. If the CNAME is fetched into cache, a next query will use that CNAME to continue from there, and miss these records next to the name.

This all does not actually explain what happened when the lookup failed for you. For that perhaps enable log-servfail: yes and see what the log error line is about the failure. What is also useful is to enable verbosity at level 4 and get the long log trace that is produced, it details what happened. It could be that the timeouts are more frequent from that location, eg. it uses a different route or destination for that IP, and it has timeouts. Unbound keeps trying for longer than dig tries, you may find it continue in logs.

@toralf
Copy link
Author

toralf commented Jun 15, 2022

I attached the output with verbosity:4 and val-log-level:2 and log-servfail: yes here.
unbound.log

@wcawijngaards
Copy link
Member

It looks like the jcloud-cdn.com. domain has nameservers that all return timeouts. No packets are returned. Also for jdgslb.com.

At the end of the trace, unbound has performed retries for all 8 for the one and all 10 for the other domain. Sent queries to the nameservers and a timeout resulted. Unbound is not yet done, it can do more probing, this can take a lot longer, with exponential backoff going to 2 minutes, until at some point it gives up. Then the log-serverfail log item would be logged with an explanation, that all the nameservers are not reachable.

So it looks like the issue is that for those two domains the nameservers do not respond, at that location.

That is
jcloud-cdn.com. with ns8.jdgslb.com. and ns7.jdgslb.com. with
183.131.197.221 114.67.160.153 114.67.121.113 101.124.23.11 183.131.197.220 114.67.160.152 114.67.121.112 101.124.23.10

And jdgslb.com. with ns4.jdgslb.com. and ns3.jdgslb.com. with
2402:db40:1570:1081::56 183.131.197.217 116.198.195.5 116.196.126.21 114.67.160.151 2402:db40:1570:1081::55 183.131.197.216 116.198.195.4 116.196.126.20 114.67.160.150

Those servers do not respond. You can find it in the log file, search for 'timeout udp' or 'DelegationPoint' printouts where the 'rtt=' values become large because of the exponential backoff.

@toralf toralf closed this as completed Jul 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@toralf @wcawijngaards