Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
kube-dns does not work due to DNS server lookup loop (/etc/resolv.conf contains 127.0.0.01 as the upstream nameserver)kubedns-masq and sidecar containers crash after doing nslookup @kubernetes/sig-network-bugs /sig area/dns #49411
Is this a BUG REPORT or FEATURE REQUEST?:
What you expected to happen:
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Linux ubuntumaster 4.10.0-19-generic #21-Ubuntu SMP Thu Apr 6 17:04:57 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
I am facing a strange issue
I then try to launch busybox using
Now when I do nslookup kubernetes it works.
ERROR: logging before flag.Parse: I0721 17:15:25.067730 1 main.go:48] Version v1.14.3-4-gee838f6
I0718 17:26:33.738368 1 nanny.go:108] dnsmasq: Maximum number of concurrent DNS queries reached (max: 150)
And at the same time I see following in sidecar
dns sidecar errorERROR: logging before flag.Parse: I0720 02:09:37.975054 1 main.go:48] Version v1.14.3-4-gee838f6
A few other details about my K8s cluster
SOS. Can someone point me in the right direction? Have spent a lot of time trying to figure this one out.
Note: Method 1 will trigger an email to the group. You can find the group list here and label list here.
Thank you all for responding. I believe I found the issue with my timeouts or rather came up with a solution that avoids one
@bowei so now its working but can you explain why its not working with systemd-resolved installed on host. You are correct there is a dns loop occuring but I am not sure with dnsmasq why that is not the case. Your help will make the picture clear to me and others. Thanks
One last thing I still see the following in sidecar logs inside docker container, but at least the crashing has stopped
ERROR: logging before flag.Parse: I0726 15:54:59.427544 1 main.go:48] Version v1.14.3-4-gee838f6
As mentioned above, please post your VM and kube-dns container /etc/resolv.conf.
root@ubuntumaster:~# cat /etc/resolv.conf
Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
127.0.0.53 is the systemd-resolved stub resolver.
run "systemd-resolve --status" to see details about the actual nameservers.
You need to delete
The loop was not eliminated, half the queries will still go into a loop, however, the other half will be successful and it may appear to be working.
I would close this issue as it appears to be a configuration issue, not a Kubernetes bug.
@bowei If I remove the nameserver 127.0.0.1 from my VM how will my node (which is the VM) use dnsmasq locally. By default Ubuntu 17.04 installs that nameserver entry when we install dnsmasq. Unless you are saying that I cannot have dnsmasq or for that matter systemd-resolved running on my VM.
Thanks @bowei for helping me with this. For others who may end up with this problem, this is what I did in Ubuntu17.04
In the end I ended up with a /etc/resolv.conf contents of
Which is my VMWare's dns server.