Containers cannot resolve DNS if docker host uses 127.0.0.1 as resolver #541

Closed
jbwyatt4 opened this Issue May 7, 2013 · 66 comments

Comments

Projects
None yet

jbwyatt4 commented May 7, 2013

I'm trying to install rails on a docker container. However the OS inside docker cannot seem to connect to the internet. I get this error message on the base (just apt-get wasn't able to connect to the internet).

sudo: unable to resolve host e62b9fa31bb9
Err http://archive.ubuntu.com quantal InRelease

Err http://archive.ubuntu.com quantal Release.gpg
Temporary failure resolving 'archive.ubuntu.com'
Reading package lists... Done
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/quantal/InRelease

W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/quantal/Release.gpg Temporary failure resolving 'archive.ubuntu.com'

W: Some index files failed to download. They have been ignored, or old ones used instead.

I've had this issue since docker 0.1 to 0.3. Running on either the host machine or in vmware. (Virtualbox does not have this issue but sometimes the kernel segfaults within the container.)

Contributor

creack commented May 7, 2013

The kernel segfault is a known bug with kernels < 3.8 (cf #407).

Regarding the internet access, can you look at your /etc/resolv.conf? It is most likely setup on a localhost. This works within the host but can't work within a container.

2 solutions:

  1. Edit the /etc/resolv.conf from the host in order to use external dns (eg 8.8.8.8, 8.8.4.4)
  2. run docker with -dns option (eg docker run -dns 8.8.8.8 -dns 8.8.4.4 base ping google.com)
Contributor

jpetazzo commented May 7, 2013

Maybe we should detect when /etc/resolv.conf is configured to use 127.*,
shouldn't we?

On Tue, May 7, 2013 at 10:00 AM, Guillaume J. Charmes <
notifications@github.com> wrote:

The kernel segfault is a known bug with kernels < 3.8 (cf #407dotcloud#407
).

Regarding the internet access, can you look at your /etc/resolv.conf? It
is most likely setup on a localhost. This works within the host but can't
work within a container.

2 solutions:

  1. Edit the /etc/resolv.conf from the host in order to use external dns
    (eg 8.8.8.8, 8.8.4.4)
  2. run docker with -dns option (eg docker run -dns 8.8.8.8 -dns 8.8.4.4
    base ping google.com)


Reply to this email directly or view it on GitHubhttps://github.com/dotcloud/docker/issues/541#issuecomment-17555383
.

Contributor

unclejack commented May 7, 2013

Perhaps we could use the proxy from docker to proxy these requests to localhost? e.g.: use 172.16.42.1 within docker containers as the DNS server and use 127.0.0.1 as the real DNS

Contributor

jpetazzo commented May 7, 2013

Well, IMHO, if we detect that the DNS server is 127.*, we should generate a
resolv.conf file containing the address of the bridge, and bind it into the
hosts.

We had lots of discussions with @creack a long time ago about this, so I
thought it was already implemented, in fact :o

Contributor

unclejack commented May 7, 2013

@jpetazzo Unfortunately dnsmasq isn't set up by default to accept requests on the IP of the docker0 bridge, even though it seems to be bound to 0.0.0.0. I've just verified whether dnsmasq allows DNS resolution when using the docker0 bridge IP address as the DNS server. Setting it up isn't an option, imo.

We could set up docker to look for an alternative resolv.conf file (e.g. /etc/resolv.docker). If 127.* is in resolv.conf, no resolv.docker exists and no -dns option was passed to docker -d or docker run, we could use public DNS and print a warning on the screen (!!! WARNING - docker is using public DNS because /etc/resolv.conf lists 127.0.0.1 as a DNS server).

Contributor

jpetazzo commented May 7, 2013

Agreed!

On Tue, May 7, 2013 at 12:44 PM, unclejack notifications@github.com wrote:

@jpetazzo https://github.com/jpetazzo Unfortunately, dnsmasq isn't set
up by default to accept requests on the IP of the docker0 bridge, even
though it seems to be bound to 0.0.0.0. I've just verified whether dnsmasq
allows DNS resolution when using the docker0 bridge IP address as the DNS
server. Setting it up isn't an option, imo.

We could set up docker to look for an alternative resolv.conf file (e.g.
/etc/resolv.docker). If 127.* is in resolv.conf, no resolv.docker exists
and no -dns option was passed to docker -d or docker run, we could use
public DNS and print a warning on the screen (!!! WARNING - docker is using
public DNS because /etc/resolv.conf lists 127.0.0.1 as a DNS server).


Reply to this email directly or view it on GitHubhttps://github.com/dotcloud/docker/issues/541#issuecomment-17565591
.

Contributor

fj commented May 21, 2013

We could set up docker to look for an alternative resolv.conf file (e.g. /etc/resolv.docker). If 127.* is in resolv.conf, no resolv.docker exists and no -dns option was passed to docker -d or docker run, we could use public DNS and print a warning on the screen (!!! WARNING - docker is using public DNS because /etc/resolv.conf lists 127.0.0.1 as a DNS server).

To whatever extent possible I think containers shouldn't know about Docker, so injecting Docker-specific knowledge inside the container is something to avoid.

If I misunderstood you and you were instead proposing having the /etc/resolv.conf be outside the containers with a fallback to hardcoded public DNS (but where docker -dns always takes priority). I think that's a good idea.

Contributor

unclejack commented May 21, 2013

@fj It was about the /etc/resolv.conf and /etc/resolv.docker which would live on the host. Docker will still manage everything on its own when setting up the resolv.conf in the container, it would just pick a DNS using the rules I've described.

Collaborator

shykes commented Jun 14, 2013

This is fixed in master. The docker daemon will detect that the host's resolv.conf points to localhost, and fallback to a public default (8.8.8.8). This can then be overridden by the admin with 'docker -d -dns'.

@shykes shykes closed this Jun 14, 2013

It seems that it’s not just the case of localhost.

mlebkowski@bob /etc> head -3 resolv.conf 
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 172.17.4.1

mlebkowski@bob /etc> ifconfig eth0 | head -2
eth0      Link encap:Ethernet  HWaddr d4:3d:7e:dc:b2:db  
          inet addr:172.17.4.41  Bcast:172.17.7.255  Mask:255.255.252.0

mlebkowski@bob /etc> ifconfig docker0 | head -2
docker0   Link encap:Ethernet  HWaddr fe:30:73:b4:77:2d  
          inet addr:172.17.42.1  Bcast:0.0.0.0  Mask:255.255.0.0

It seems that docker uses the same network as the host. I can’t really tell if this is a docker issue or my network configuration. Should I reopen the ticket?

Member

tianon commented Nov 30, 2013

Well, those networks are indeed overlapping, but that looks like it might be a separate issue, since Docker's automatic network detection code should've avoided exactly that, as far as I remember (it's been a while since I looked at that part of the code).

I believe the exact DNS fix described here was that if your DNS server is anything in a "local" block, it gets remapped to 8.8.8.8, so even in your case it should've been remapped, as far as my understanding goes. Perhaps someone with more knowledge of that specific part of the code will chime in here. :)

cattz commented Dec 10, 2013

Maybe a better option, other than remap the nameserver to a public one would be to have an way to configure the 'default' nameserver for containers. I can get DNS working fine when I run it with the -dns flag. The problem I'm facing is how to have stuff installed when using docker build as apt-get/yum cannot resolve the repos URLs. Having a public nameserver doesn't work for me, since I'm inside a corporate netrwork where DNS traffic seems to be -somehow- restricted.

cattz commented Dec 10, 2013

I can reply to my own question: dotcloud#815

(..) in order to use dns server side (i.e. like doing -dns on each run automatically), you can start docker daemon with -dns x.x.x.x.
Ex: docker -d -D -dns 8.8.8.8 -dns 8.8.4.4

Puzzled newbie here... I am running Docker inside of a Vagrant vm

Yesterday I was able to docker build . an image successfully, today trying a different image (both are FROM ubuntu:12.04) I am getting the "Temporary failure resolving 'archive.ubuntu.com'" errors when the image tries to do apt-get stuff

I tried adding RUN echo "nameserver 8.8.8.8" >> /etc/resolv.conf to the Dockerfile but it errors ("returned a non-zero code: 2")

My /etc/resolv.conf in the vm looks like:

# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 10.0.2.3

What am I supposed to do to make it work?

and... rebooting the vagrant vm fixed it

Contributor

tamsky commented Jan 6, 2014

This is fixed in master. The docker daemon will detect that the host's resolv.conf points to localhost, and fallback to a public default (8.8.8.8). This can then be overridden by the admin with 'docker -d -dns'.

The 'fix' above excludes all production environments which deliberately run a caching nameserver on 127.0.0.1.

Our resolv.conf reads:

nameserver 127.0.0.1
nameserver <non-local-ip-of-fallback-resolver#1>
nameserver <non-local-ip-of-fallback-resolver#2>

Is there a way to support environments which run a per-host caching-only dns recursive resolver?

I'm experiencing the exact same issue as tamsky -
I'm running 12.04.3 with kernel 3.8.0-35 and docker version 0.7.5 (existed in docker version 0.7.3 as well)

My company does a caching nameserver at 127.0.0.1 and my resolv.conf reads the same as tamsky's.

I've appended 'nameserver 8.8.8.8' to my resolv.conf, but it appears docker never checks that far down?

In my honest opinion, this kills the portability factor of docker, as currently the daemon has to be restarted using a specified dns server. It looks like there have been several issue threads discussing this problem - what is the possibility/probability of a docker.conf where we can specify instance variables like dns?

Thanks!

Member

tianon commented Jan 16, 2014

@badger32d just as a point of interest, if all you need to "fix" this in your environment is a configuration file to specify -dns for the daemon, then "/etc/default/docker" is just such a thing (provided you use Upstart or SysV, which if you're on 12.04 you do).

Just put something like this inside:

DOCKER_OPTS="-dns 8.8.8.8 -dns 8.8.4.4"

Then service docker restart or restarting your system completely will make sure Docker starts up with -dns as specified.

So it's not exactly a solution to all the issues, but it's certainly a reasonable way to specify the workaround in a simple and persistent way.

Tianon: I didn't realize that was a possibility, thanks! That should solve it.

Contributor

tamsky commented Jan 19, 2014

@badger32d 's solution will not work on our network.
Our machines do not have direct access to the full internet, but they can reach a nearby recursive dns resolver.

I'm not sure what our workaround will be but it might involve having named listen on the docker0 interface, and setting the DOCKER_OPTS="-dns 172.17.42.1"

Contributor

crosbymichael commented Jan 20, 2014

@tamsky yes, you need to have your resolver bind on the docker0 bridge. Then pass the -dns flag to the docker daemon with the bridge ip. This ill make all container's created use the bridge as the resolver.

Contributor

tamsky commented Feb 2, 2014

It sure would be nice if these issues could be addressed:

  1. In order for our localhost bind nameserver to actually bind()/listen() on the bridge interface, dockerd must have already started, and successfully created and configure(number) the bridge interface. This ordering requirement is sub-optimal.
  2. There is no deterministic address that we can configure bind to listen() on, as it's dynamic.

Can we, as part of our system networking startup configuration, create and configure the bridge interface before dockerd starts?

If so, that would resolve both issues.

Member

tianon commented Feb 3, 2014

@tamsky Absolutely! If you create/configure the docker0 interface before starting Docker, Docker will pick up the network settings from the interface instead of making them up (for example, using the IP/mask on the bridge instead of trying to find a suitable one).

You can also use -b to change the bridge that Docker attaches to, or --bip to just change the bridge's IP/mask:

$ docker -h
Usage of docker:
...
  -b, --bridge="": Attach containers to a pre-existing network bridge; use 'none' to disable container networking
  --bip="": Use this CIDR notation address for the network bridge's IP, not compatible with -b
...

I have installed docker on 1) Ubuntu 12.04 and 2) CentOS 6.4. I did this on my work laptop connected to our corporate VPN, as I was trying to demonstrate docker for a potential production use.

However, I cannot connect to internet (if I had to run a mvn install) or yum -y update inside the image.

Do you guys think it is related to the DNS issue?

dodev commented Feb 25, 2014

@seshendra I think I'm facing the same problem:

  • I'm running ubuntu quantal
  • connected to the corporate VPN the containers have network access, can resolve hostnames;
  • on wlan0 the containers still have network access, but can't resolve names.

And yes, I think it's DNS related :)

@dodev for now I solved this by doing two things

  1. dnsmasq (where I use custom DNS servers)
  2. I used bridged connection instead of NAT, on the host virtualbox VM (this also solved inter container communication issue that had earlier).

I'm not exactly sure which one solved the DNS issue --- but working for now on CentOS. Haven't tried on Ubuntu yet.

@seshendra : Did you add any changes to /etc/resolv.conf file or set any env variable for this?
or Proceed with Mask DNS ?
https://wiki.debian.org/HowTo/dnsmasq

@shafi - I've upgraded to docker 0.9 and everything seems to be working fine both on centos and Ubuntu.

Thanks!
Seshendra Nalla.

On Mar 18, 2014, at 10:18 PM, shafi notifications@github.com wrote:

@seshendra : Did you add any changes to /etc/resolv.conf file or set any env variable for this?
or Proceed with Mask DNS ?
https://wiki.debian.org/HowTo/dnsmasq


Reply to this email directly or view it on GitHub.

wt commented May 16, 2014

I am going to use this trick on the docker host:

  1. Manually create docker-bridge
  2. sysctl net.ipv4.conf.docker-bridge.route_localnet=1
  3. iptables -t nat -I PREROUTING -d 192.168.0.1 -p tcp --dport 25 -j DNAT --to-destination 127.0.0.1:25

The 192.168.0.1 is the docker default gateway. You can now do "nc 192.168.0.1 25" to get the mail server. The same trick can be probably be used for DNS. You'll have to use "-p udp" instead, of course. I haven't tried this with DNS, so YMMV. And i guess you could take it one step further and add a DNAT rule inside the container to direct requests to 127.0.0.1:25 to 192.168.0.1:25 if you are hell bent on having the container talk to 127.0.0.1 instead of 192.168.0.1.

Quasaur commented Aug 14, 2014

on Debian 7.6 i have the same issue...
(1) Running docker 1.0.0
(2) /etc/default/docker.io has the DOCKER_OPTS commented with the DNS servers.
(3) my /etc/resolv.conf (generated by Connection Manager) points to the localhost as name server (though i believe that, thought bind9-host & dnsutils are installed, bind9 isn't).
(4) i rewrote /etc/resolv.conf, placing my ISP's nameservers first, the docker-supplied nameservers 2nd, and the localhost nameserver last:


Generated by Connection Manager

nameserver 209.18.47.61
nameserver 209.18.47.62
nameserver 8.8.8.8
nameserver 8.8.4.4
nameserver 127.0.0.1
nameserver ::1


do i need bind9 (or some something slimmer) installed to resolve addresses for docker? If so, why doesn't everything else on my system seem to resolve addresses with no issue?

Contributor

cpuguy83 commented Aug 14, 2014

@Quasaur containers can't reach that 127.0.0.1 address (since they have their own 127.0.0.1).

seemant commented Oct 30, 2014

I am having same issue.. Trying to set up 3 docker instances for a test-automation. Each instance has a local IP. The automation test-bed assumes that there are 3 machines with given IP. It doesn't care if the 3 machines are dockers or VM instances or physical machines. So we do not want to use docker commands to set up the machines for automation.

We create a base docker ubuntu image with SSH service enabled. Then we use SSH commands in the robotframework to set up the machines with required packages. Along the way we need to setup /etc/hosts file so that apps running on one machine can discover the apps on the other. This used to work when we started using docker first time.. With recent changes its broken and we cannot edit the /etc/hosts file.

So for work around I thought I would try to install dnsmasq and change the /etc/resolv.conf file to point to 127.0.0.1, then with dnsmasq options I could specify some other file for static host name to ip bindings. While this works if the apps inside the docker explicitly send query to 127.0.0.1, it doesn't work when apps rely on gethostbyname API(s) provided by the system. So, I thought of modifying resolv.conf then realized that .. it cannot be modified and it set to Google DNS servers 8.8.8.8 and 8.8.4.4. I thought we can modify the resolvconf "head" file and restart resolvconf service.. But wait.. the service complains that the /etc/resolv.conf is not a link to /var/run/resolvconf/resolve.conf. So there goes that approach out the window.

So now what is left.. After some more googling I find this solution here:
https://sites.google.com/site/randomt3ch/home/how-to-add-new-dns-entry-in-docker-resolv-conf. And I thank my stars.. But wait!!.. Now the docker build fails. If I put 127.0.0.1 as first dns server as suggested by the file, the packages specified in the Dockerfile cannot be downloaded. So now what?

Brilliant minds here suggest to use sudo docker -d dns to change the DNS server of a a running docker instance.. That means automation tests need to have 2 approaches, now one for docker based test-bed and one for VM based test-bed.. We can do that I guess. but I am not seeing the point of all these restrictions in a docker? If its security that is bothering,, then can we have an option to build a "non-secure" image that allows changes to anything?

Thanks

Contributor

mcandre commented Mar 7, 2015

I experienced the same error today when I switched WiFi networks. I was able to fix this with boot2docker restart. (I'm using Mac OS X for my host.)

rektide commented May 27, 2015

I flit between some essential internal DNS servers, and am on Ubuntu. With these tweaks, their dnsmasq set-up & run by their network-manager serves my Docker containers-

  • net.ipv4.conf.docker0.route_localnet = 1 in /etc/sysctl.conf (big thanks @wt!) (and restart procps)
  • put listen-address=0.0.0.0 in a new file /etc/NetworkManager/dnsmasq.d/listen-all.conf (and restart network-manager)
  • set DOCKER_OPTS="--dns=172.17.42.1" in /etc/default/docker (and restart docker)

@haugene haugene referenced this issue in haugene/docker-transmission-openvpn May 28, 2015

Closed

Self-built docker unable to resolve host #19

I also experienced the issue mcandre reported on OSX when switching from WiFi to 3G after starting boot2docker on WiFi. The suggested workaround boot2docker restart worked for me.

I was also able to fix this issue with boot2docker restart.

mmahalwy commented Dec 7, 2015

boot2docker restart is not working for me. Running into the same problem with ubuntu:
Could not resolve 'archive.ubuntu.com'

My host /etc/resolv.conf points to 192.168.1.1, and I have set dnsmasq on docker0 (172.17.42.1) interface to use local dns resolver for containers (similar to http://docs.blowb.org/setup-host/dnsmasq.html). now, in host machine, internet dns queries are resolved in both 192.168.1.1 and 172.17.42.1 interfaces. I have to use --dns 172.17.42.1 option to docker containers, since dnsmasq on docker0 hosts local dns entries of other containers. But the problem here is that public URLs archive.ubuntu.com is not resolving and so package update fails inside container

Docker Host: (/etc/resolv.conf)
search lan
nameserver 192.168.1.1

Inside container: (/etc/resolv.conf)
nameserver 172.17.42.1
search lan

Is there any workaround for this case

@tayzlor tayzlor referenced this issue in Capgemini/Apollo Feb 18, 2016

Closed

Registrator fails on reboot #617

Why can't Docker figure out my DNS is configured to 192.168.0.1?

We rely on some special DNS rules on our own DNS server. The PC points to that DNS server, however because /etc/resolv.conf contains 127.0.0.1, it will default to Google DNS. So our DNS rules don't work and stuff breaks during development.

Contributor

mavenugo commented Apr 1, 2016

@roelvanduijnhoven as mentioned here : docker#541 (comment) this is the expected behavior when /etc/resolv.conf contains 127.0.0.1. But you can use --dns option while running the daemon or when running the container using docker run and pass 192.168.0.1. Will that work for you ?

That does work, I already fixed it this way on my machine.

But this is not ideal. You will need to reconfigure this setting every time your laptop moves to another location as the DNS location will change.

raarts commented Apr 30, 2016

Maybe this is useful to others, I solved it as follows:

  • Create the docker0 bridge beforehand (and add iptables NAT rule)
  • give it the same ip as docker would (in my case 172.17.0.1/16)
  • bind a DNS resolver to that ip address only (I used consul, but dnsmasq or BIND would do too)
  • put this ip in /etc/resolv.conf as nameserver
  • start docker and tell it to use that 172.17.0.1 as DNS server

This makes both regular containers as well as those on user-defined networks just work.

Contributor

denibertovic commented Jun 3, 2016

I understand that it's desirable to rewrite /etc/resolv.conf if it only has 127.0.0.1 listed because the containers are isolated in their own network. But if I'm using net="host" isn't this behavior a bug?

Specifically I think I'm seeing a race because one container get's started with the correct /etc/resolv.conf from the host and the other one (also net:"host") get's a rewritten /etc/resolv.conf with public dns's listed.

Contributor

justincormack commented Jun 3, 2016

There is an outstanding issue to change this behaviour in the case of net=hostsee #22853

Contributor

denibertovic commented Jun 3, 2016

Also related: docker#22408

skirdey commented Jun 10, 2016

This helped in my case:

config.vm.provision :shell, :inline => "echo 'nameserver 8.8.8.8' | sudo tee /etc/resolv.conf &> /dev/null", :privileged => true

I have just hit this issue. Hard.
I really think that in that case (DNS set to 127.0.0.1) docker should just proxy those requests to the outer "localhost". It's what I expected. Docker does a lot of things, why not this?! Everything else just sucks, sorry.

Contributor

cpuguy83 commented Jul 7, 2016

@flypenguin please upgrade to 1.11 and try again. Thanks!

Hm. Tested on 1.9.1 and 1.11.2, didnt work in both ...

sanimej commented Jul 7, 2016

@flypenguin Yes, forwarding the queries to the host's 127.0.0.1 is not supported now. You can use the --dns option on container run. Will that work for you ?

flypenguin commented Jul 11, 2016

[Update]

Okay, solved it it seems. Ubuntu 16.04 installed - for whatever reason - the lxc packages, which contain a local dnsmasq set up which somehow prevents DNS to be used in the containers. Seems I have to read up on what that is, but uninstalling lxc* packages did the trick.

Super weird.

[Original post]

It might - but I think I hit another problem. I opened all DNS servers on the "public" network interfaces now, and on some machines it "just works", on others it doesn't. It actually seems to be distribution dependent.

On Ubuntu 16.04:

root@ip-10-10-11-12:~# docker run --rm -ti busybox
/ # ping 10.10.11.12
PING 10.10.11.12 (10.10.11.12): 56 data bytes
64 bytes from 10.10.11.12: seq=0 ttl=64 time=0.096 ms
64 bytes from 10.10.11.12: seq=1 ttl=64 time=0.091 ms
^C
--- 10.10.11.12 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.091/0.093/0.096 ms
/ # cat /etc/resolv.conf
# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 10.10.11.12
search eu-central-1.compute.internal
/ # ping heise.de
^C
/ # nslookup heise.de
Server:    10.10.11.12
^C
/ # 
/ # root@ip-10-10-11-12:~# dig @10.10.11.12 heise.de
[cut]
;; ANSWER SECTION:
heise.de.       3161    IN  A   193.99.144.80
[cut]
;; Query time: 0 msec
;; SERVER: 10.10.11.12#53(10.10.11.12)
;; WHEN: Mon Jul 11 11:59:28 UTC 2016
;; MSG SIZE  rcvd: 53

root@ip-10-10-11-12:~# 

On Ubuntu 15.10:

root@ip-10-10-11-10:~# docker run -ti --rm busybox
/ # ping heise.de
PING heise.de (193.99.144.80): 56 data bytes
64 bytes from 193.99.144.80: seq=0 ttl=246 time=2.197 ms
64 bytes from 193.99.144.80: seq=1 ttl=246 time=1.932 ms
^C
--- heise.de ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 1.932/2.064/2.197 ms
/ # nslookup heise.de
Server:    10.10.11.10
Address 1: 10.10.11.10 ip-10-10-11-10.eu-central-1.compute.internal

Name:      heise.de
Address 1: 2a02:2e0:3fe:1001:302:: redirector.heise.de
Address 2: 193.99.144.80 redirector.heise.de
/ # root@ip-10-10-11-10:~# ^C
root@ip-10-10-11-10:~#

Summary:

  • Ubuntu 15.10:
    • works.
  • Ubuntu 16.04:
    • freshly booted, docker 1.11.2, Ubuntu 16.04
    • in container: host IP is set as nameserver in /etc/resolv.conf
    • in container: can ping the host IP from within container
    • in container: cannot resolv DNS queries using host IP as DNS
    • host DNS working perfectly fine

Both hosts should have exactly the same configuration (I use the same puppet rules for both), and I don't know of any difference except some installed software or IP address out of my head.

Contributor

denibertovic commented Sep 12, 2016

This might be related with the issue I was seeing and posted above but was not really given much attention. Namely, I think I was seeing a race condition because the issue was not happening on different OS version but on the same OS in different containers. Some got the correct /etc/resolv.conf some did not. I have not checked with the latest release.

EDIT:

I did some quick testing on 1.11.2: docker run -it --net="host" debian:jessie /bin/bash -c "cat /etc/resolv.conf".

I ran the command a couple of times and I can't seem to reproduce like I was able before. So at least the perceived race is gone.

Will there be a real fix to this? Such that if /etc/resolve.conf points to localhost, the docker daemon will forward dns requests to localhost? (as opposed to forwarding them to google's dns servers)

Contributor

denibertovic commented Sep 14, 2016

@rage-shadowman Are you talking with "--net=host"? Cause that appears to to be working with 1.11.2. If you're not talking about "--net=host"... I don't see how that could work.

rage-shadowman commented Sep 14, 2016

No, I mean using the default bridge network. You are already filtering out localhost from /etc/resolv.conf. Why not simply replace it with the ip address of the host? My current hack is to do this in my .bashrc:

device-ip ()
{
    ip addr show dev "$1"|grep ' inet '|sed 's/ *inet \([0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+\)\/[0-9]\+.*/\1/'
}

get-docker-dns-opts ()
{
    WLAN_IP="$(device-ip wlan1)"
    if [ -n "$WLAN_IP" ]
    then
        echo "-dns $WLAN_IP"
    fi

    ETH_IP="$(device-ip eth1)"
    if [ -n "$ETH_IP" ]
    then
        echo "-dns $ETH_IP"
    fi

    # also add google dns servers
    echo "-dns 8.8.8.8"
    echo "-dns 8.8.4.4"
}

export DOCKER_OPTS="$(get-docker-dns-opts)"

However, this is an ugly hack with its own issues. It currently works for me in my dev environment but a real solution which points dns requests to the host machine would be much preferable.

PS - I will admit to having very little knowledge of the inner workings of docker. However, to my uneducated mind this seems like something that should be workable.

Member

thaJeztah commented Sep 21, 2016

Also see docker#23910

@bagage bagage referenced this issue in arno01/steam Oct 25, 2016

Closed

DNS requests flood #6

margh commented Jan 5, 2017

FWIW

On Ubuntu 15 with a bind9 setup I hit this while trying to build containers where you can't specify a dns flag.

Docker seemed to be clipping 127.0.0.1 out of the list of nameservers but not adding in google's as a replacement so I simply added nameserver 8.8.8.8 at the bottom of my host /etc/resolve.conf and everything came good.

moepot commented Jan 25, 2017

I agree with @rage-shadowman and @flypenguin . There should be a better solution than to simply point to the Google nameservers, because one wants to use his own network's nameserver.

Why is this issue closed? Are you going to fix this problem?

Member

thaJeztah commented Jan 25, 2017

@moepot you can use your own network's name server; the problem in this case is that a container has it's own network namespace; 127.0.0.1 is localhost inside the container, not localhost of the host that the container runs on. Other IP-addresses on the local network should work.

I think everybody is aware why that is not working. I also think most people here (definitely including myself) think docker should just make this possible ;)

moepot commented Jan 25, 2017

@thaJeztah thanks for your answer. I understand the technical details, but I'm not very happy with the current solution. Of course I can set my own name server with the --dns option, but in my opinion this is a dirty solution.

So my question is: Do we have to live with the --dns thing, or is there any possibility for another solution from Docker's point of view?

sanimej commented Jan 25, 2017

@moepot @flypenguin @margh @thaJeztah Fix for this is available in docker master and will be included in 1.14. With Docker DNS server we can differentiate between host's loopback vs container's loopback (passed though --dns) and handle it correctly. Note that it works only for containers on user defined networks.

Member

thaJeztah commented Jan 25, 2017

@sanimej do you have the PR that vendored it into docker?

sanimej commented Jan 25, 2017

@thaJeztah Its docker#29891. With this change there is an issue in upgrading from 1.12.x to 1.14. Its fixed in libnetwork. I will vendor it shortly.

macmpi referenced this issue in volumio/Build Mar 3, 2017

Have just installed docker-engine 17.06.1-ce build 874a737

Same problem. Running a host (VMware) instance of dnsmasq that listens on 127.0.0.1 and resolv.conf reflects this. Docker containers cannot resolve through this server. I was under the impression that this had been fixed.

Ideas welcome.

One possible thought here is to alias a second, non-routable IP onto the host NIC and run the dnsmasq instance there, thereby giving it a non-localhost listening address. Has anyone tried this?

themotu commented Dec 14, 2017

for me containers start and have working resolution, after random amount of time all dns resolution fails

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment