Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot install resolvconf on Ubuntu 12.04 containers #1297

Closed
sridatta opened this issue Jul 25, 2013 · 48 comments

Comments

@sridatta
Copy link
Contributor

@sridatta sridatta commented Jul 25, 2013

Docker currently bind-mounts to the container's /etc/resolv.conf. This causes problems when installing utilities such as resolvconf/dnsmasq/dhcpd, because /etc/resolv.conf needs to overwritable. Here's the output when trying to run resolvconf:

lsattr: Inappropriate ioctl for device While reading flags on /etc/resolv.conf
ln: cannot remove `/etc/resolv.conf: Device or resource busy

However, /etc/resolv.conf is a sensible default location. Can we instead make the mount location configurable?

@creack

This comment has been minimized.

Copy link
Contributor

@creack creack commented Jul 26, 2013

Docker mount-bind the /etc/resolv.conf file in order to have consistent dns with the host. Maybe we can add an option to skip this step?

@creack

This comment has been minimized.

Copy link
Contributor

@creack creack commented Jul 26, 2013

@vieux

This comment has been minimized.

Copy link
Collaborator

@vieux vieux commented Jul 31, 2013

otherwise isn't it possible to copy it, instead of mount-binding it in read-only ?

@creack

This comment has been minimized.

Copy link
Contributor

@creack creack commented Jul 31, 2013

@vieux this would modify the file system and polute the diff.
@sridatta As a workaround, you can move the docker's /etc/resolv.conf and then install resolvconf.

@sridatta

This comment has been minimized.

Copy link
Contributor Author

@sridatta sridatta commented Jul 31, 2013

Perhaps docker could create an ro layer that contains a copy of /etc/resolv.conf? That would avoid polluting the image diff.

@thijsterlouw

This comment has been minimized.

Copy link
Contributor

@thijsterlouw thijsterlouw commented Aug 8, 2013

I also just ran into this issue when trying to add custom internal repositories that need to be resolved using our own nameservers.

@creack what do you mean with 'move the docker's /etc/resolv.conf and then install resolvconf" ?
@sridatta that sounds like a logical solution. Did you get it working in the meantime?

@creack

This comment has been minimized.

Copy link
Contributor

@creack creack commented Aug 8, 2013

@thijsterlouw Nevermind, I though we could move the file but apparently not :(

@ghost ghost assigned crosbymichael Aug 15, 2013
@jpetazzo

This comment has been minimized.

Copy link
Contributor

@jpetazzo jpetazzo commented Aug 15, 2013

There is a workaround. Make /etc/resolv.conf a symlink (to e.g. /etc/resolv.conf-docker, which has to exist as well). Docker's bind-mount will follow the symlink. You can then delete the symlink and replace it with whatever you want.

@crosbymichael

This comment has been minimized.

Copy link
Member

@crosbymichael crosbymichael commented Aug 15, 2013

@jpetazzo

Are you not a fan of a ro layer for this file?

Michael Crosby

On Aug 14, 2013, at 5:58 PM, Jérôme Petazzoni notifications@github.com wrote:

There is a workaround. Make /etc/resolv.conf a symlink (to e.g. /etc/resolv.conf-docker, which has to exist as well). Docker's bind-mount will follow the symlink. You can then delete the symlink and replace it with whatever you want.


Reply to this email directly or view it on GitHub.

@fkautz

This comment has been minimized.

Copy link
Contributor

@fkautz fkautz commented Aug 15, 2013

The symlink could exist on a ro layer, where the symlink points to an ephemeral writable layer containing the configuration. e.g. /dockertmp/resolv.conf where /dockertmp is tmpfs. This would allow users to customize resolv.conf while simultaneously preventing container authors from breaking the contract.

@jpetazzo

This comment has been minimized.

Copy link
Contributor

@jpetazzo jpetazzo commented Aug 15, 2013

I have mixed feelings about solutions which would be AUFS-specific, because they will make everyone's life harder when we roll out support for other storage backends.

That being said, I want to find a solution :-) But I feel like I don't understand the problem well enough.

@sridatta and @thijsterlouw, can you give some details about what you're trying to achieve? For instance, @thijsterlouw, I don't understand why you can't specify the DNS server on the Docker host, and let it propagate to the containers...?

@thijsterlouw

This comment has been minimized.

Copy link
Contributor

@thijsterlouw thijsterlouw commented Aug 15, 2013

@jpetazzo : I am creating a Dockerfile for internal use. During build-time of the image, I want to pull packages from internal repositories. Those repositories need to be resolved via custom nameservers. I am using the stock SL-6.4 image as base. I know how to propagate them to containers (though that is a bit limiting in itself with the -dns command line option), but I don't know how to use them at build-time.

I will try the work-around with the symlink; I assume in your workaround you mean I need this symlink in my base-image, not on the host? In the container we can ofcourse not change it.

@jpetazzo

This comment has been minimized.

Copy link
Contributor

@jpetazzo jpetazzo commented Aug 15, 2013

Hmm, sorry if this is a dumb question, but: why can't you configure the
host to use your custom nameservers?
Is it because the custom nameservers only resolve internal names, and not
external ones?

(My first workaround was, indeed, to set the symlink in your images; not
necessarily base image, though. It can be any layer.)

@drewcsillag

This comment has been minimized.

Copy link

@drewcsillag drewcsillag commented Aug 15, 2013

I'm also in this same boat. I'm not always on VPN, so I don't want my host settings to include the custom nameservers, but I do fire up VPN to do the builds of the images. To make things more fun, the custom resolvers seem not to be able to resolve the names of standard debian repos (for example), so I have to switch resolv.conf settings during the build of the image. Personally, I'd love a switch that would say "just don't touch resolv.conf, I'll do it myself". Using the LXC template stuff I'm doing in #1383 allows me to mount /etc/resolv.conf out of the way, but something else is "conveniently" overwriting whatever I put there for subsequent RUN commands (seems like it started in the last week or so). graph.go looks to be the culprit, but I've not had enough time to fully debug it.

@sridatta

This comment has been minimized.

Copy link
Contributor Author

@sridatta sridatta commented Aug 15, 2013

In my case, I've gotten around it using the symlink hack. But my containers need to have custom resolvers because we use DNS for service discovery/registry and I don't want to pollute the host with such things.

@jpetazzo

This comment has been minimized.

Copy link
Contributor

@jpetazzo jpetazzo commented Aug 15, 2013

Okay. And what about using -dns ?
Would that help?
Or do you want to have different DNS resolvers in different containers as
well?

@thijsterlouw

This comment has been minimized.

Copy link
Contributor

@thijsterlouw thijsterlouw commented Aug 20, 2013

I must be overlooking something, but I dont get the symlink hack. If i try to use the ubuntu or sl6 base image and in my Dockerfile then specify a RUN command to symlink the resolv file to something else, I will always get permission denied. Also if I try to mount a tempfs. If you happen to have a working Dockerfile example, I would be really happy!

@jpetazzo

This comment has been minimized.

Copy link
Contributor

@jpetazzo jpetazzo commented Aug 20, 2013

Ah, I think that /etc/resolv.conf has to be a link file in the first place
(i.e. at initial image creation), otherwise you won't be able to change it
from Docker itself (since instantiating a container from the image will
always bind-mount /etc/resolv.conf, preventing you from replacing it with a
symlink).

I would suggest the following hack:

  • docker run -d myimage true
  • note container ID
  • go to /var/lib/docker/containers//rootfs/etc
  • touch resolv.conf-docker ; rm resolv.conf ; ln -s resolv.conf-docker
    resolv.conf
  • cd ~
  • docker commit myimage

On Tue, Aug 20, 2013 at 7:53 AM, Thijs Terlouw notifications@github.comwrote:

I must be overlooking something, but I dont get the symlink hack. If i try
to use the ubuntu or sl6 base image and in my Dockerfile then specify a RUN
command to symlink the resolv file to something else, I will always get
permission denied. Also if I try to mount a tempfs. If you happen to have a
working Dockerfile example, I would be really happy!


Reply to this email directly or view it on GitHubhttps://github.com//issues/1297#issuecomment-22950619
.

@thijsterlouw

This comment has been minimized.

Copy link
Contributor

@thijsterlouw thijsterlouw commented Aug 20, 2013

thanks @jpetazzo,I think I almost understand. But rootfs inside the container does not exist untill you commit the container? So if I just start a container from an image I do not have the rootfs/etc/... yet.

@drewcsillag

This comment has been minimized.

Copy link

@drewcsillag drewcsillag commented Aug 20, 2013

On Tue 20 Aug 2013 04:35:25 PM EDT, Thijs Terlouw wrote:

thanks @jpetazzo https://github.com/jpetazzo,I think I almost
understand. But rootfs inside the container does not exist untill you
commit the container? So if I just start a container from an image I
do not have the rootfs/etc/... yet.


Reply to this email directly or view it on GitHub
#1297 (comment).

Once you get this working, if you could write up a quick doc (or email
is fine too) explaining what you did, that'd be a great help to me too!
I sorta get what's supposed to happen, but not quite enough.

@jpetazzo

This comment has been minimized.

Copy link
Contributor

@jpetazzo jpetazzo commented Aug 21, 2013

Erm, sorry guys, I was dead wrong!
As hinted by @drewcsillag, there is some magic in graph.go, to enforce that some files/directories exist and are of the correct type; and that includes resolv.conf:
https://github.com/dotcloud/docker/blob/d627ff969735d44201fc88539d388ad87f102cda/graph.go#L204

For @sridatta use case, I think that -dns should work well enough.

However for @thijsterlouw use case (switching DNS servers during the build), it will be tricky.
Let's see the options.

  • You could compile your own docker, commenting out the line above, so that /etc/resolv.conf just happens to be a normal file.
  • You could run a dnsmasq (either on your host, or in a container), with a "clever" configuration that would "just work" and resolve everything properly (if you're not familiar with dnsmasq, give more details about your setup and I can give some guidance).
  • Something else :-)
@thijsterlouw

This comment has been minimized.

Copy link
Contributor

@thijsterlouw thijsterlouw commented Aug 21, 2013

thanks @jpetazzo ! that probably explains the errors I was getting about not being able to find /etc/resolv.conf in the container, even though I was looking at the file.

Actually I am running Ubuntu on my host, so I also have dnsmasq on my host. In my /etc/resolv.conf I have an entry which points to localhost (127.0.1.1). Perhaps that is why I am getting the default /etc/resolv.conf with the google nameservers. Anyway I am already running head version of Docker, so I might as well patch it and see if I can make progress that way. Why is resolv.conf treated specially? I'll just try patching now and will report back later.

@jpetazzo

This comment has been minimized.

Copy link
Contributor

@jpetazzo jpetazzo commented Aug 21, 2013

/etc/resolv.conf is handled specially because in 99% of the cases, we want
to make sure that /etc/resolv.conf points to working name servers
(generally, those of your host machine). Embedding the DNS servers within
the image wouldn't work*. However, we don't want to change /etc/resolv.conf
each time you start a container, because that would pollute the "diff"
between the image and the container. So we use that trick, to "import" the
/etc/resolv.conf file of the host, without actually changing it physically
in the container.

*One other option was to use a special entry for the DNS server, and then
have Docker hijack the communication if needed. We considered it, but
finally decided against it because it means that when you start a
container, if /etc/resolv.conf doesn't have the default resolvers, you have
to wonder: is it by purpose (because the container author/runner knows what
she's doing) or by mistake? We finally thought that the bind-mount was a
clever solution. Or, maybe, the "least worse" option :-)

@drewcsillag

This comment has been minimized.

Copy link

@drewcsillag drewcsillag commented Aug 22, 2013

It would be nice though to have a "just don't touch the stinking resolv.conf file" option in the cases where you do need to mess with it during a build or you in fact do want 127.0.1.1 to be your name server or the other cases where docker 'helps'. We used #1383 partly for this purpose -- we killed the bind-mount, ADDed an installer script in the Dockerfile and RUN'd the script. So the resolv.conf file is written in the installer script and sticks for the length of it's execution -- actually we write resolv.conf twice in our installer script, but you get the idea. The resolv.conf file is then empty on subsequent RUN commands, but it works. Either way, a "stop 'helping' /etc/resolv.conf" option would be great.

Another, wrong way-but-works-way to partly work around this is to remove /var/lib/docker/graph/_tmp/docker/etc/resolv.conf which will keep it out of the way. Since it's empty, you can recreate it easily enough later.

As an aside, having some switch to add a "search" line to /etc/resolv.conf would be super handy.

@thijsterlouw

This comment has been minimized.

Copy link
Contributor

@thijsterlouw thijsterlouw commented Aug 23, 2013

thanks @jpetazzo. Perhaps you can also take a look at my pull-request. It caused me to have the default google-nameservers and not understand your remarks about 'why do you not use the host dns servers'.

+1 regarding the search line to resolv.conf. That's what I meant earlier when I said it is a bit limiting the way it is (because you can only set nameservers now). Also +1 for the option to tell Docker not to mess with the resolv.conf

@drewcsillag

This comment has been minimized.

Copy link

@drewcsillag drewcsillag commented Aug 23, 2013

One other thing to consider is that, while resolv.conf files are usually simple affairs, they aren't always. If you look at the man page for resolv.conf you'll see a bunch of stuff. There are a few options on how to address this:

  • try to provide some form of syntax to produce all possible resolv.conf things
  • try to provide some lesser percentage and say you can't do these to the rest and/or require that they do some local hack to make it work
  • provide some proper way for users to manage it themselves

Even though adding a search option is an attractive idea, in retrospect, if you need that, you're probably in a good position to just manage resolv.conf yourself and having to add configuration and various options to coax docker into producing a resolv.conf file you already know how to write yourself seems wrong-headed.

Given this: I guess a simple "don't touch /etc/resolv.conf" option is my current preference, or maybe something where you can pass the resolv.conf to docker build/run/-d.

@jpetazzo

This comment has been minimized.

Copy link
Contributor

@jpetazzo jpetazzo commented Aug 29, 2013

Let's recap a little bit!

The current filesystem implementation puts /etc/resolv.conf, /.dockerinit,
and a few other mandatory files/devices/directories into a read-only layer.
This:

  • ensures that they are present in the final filesystem of the container,
    "no matter what"
  • keeps them out of the "rw" layer, meaning that they won't pollute the
    output of "docker diff" nor pushed layers

However, this approach has drawbacks:

  • it prevents people from altering /etc/resolv.conf if they need to
  • it is AUFS-specific

Yesterday, while discussing with @vieux and @crosbymichael, an idea popped
up: we could create those files/devices/directories in the container, but
exclude them from "docker diff" and "docker push" if they haven't been
modified. This means that:

  • those files will be automatically injected if they are missing from an
    image
  • the container can alter them (which can result in breakage; but well, if
    you "rm -rf /" it will also result in breakage...)
  • (Docker could display warnings if you run a container where those
    elements are altered)
  • those files will not pollute the output of docker diff (and the
    .dockerinit binary won't add its weight to all the pushed layers!) as long
    as they haven't been modified
  • (Pushing an image where those files haven been modified could also
    display a warning and/or require some -force flag)
  • this approach is independent of the underlying storage layer

Would do you guys think?

/cc @creack

On Fri, Aug 23, 2013 at 6:50 AM, Drew Csillag notifications@github.comwrote:

One other thing to consider is that, while resolv.conf files are usually
simple affairs, they aren't always. If you look at the man page for
resolv.conf you'll see a bunch of stuff. There are a few options on how to
address this:

  • try to provide some form of syntax to produce all possible
    resolv.conf things
  • try to provide some lesser percentage and say you can't do these to
    the rest and/or require that they do some local hack to make it work
  • provide some proper way for users to manage it themselves

Even though adding a search option is an attractive idea, in retrospect,
if you need that, you're probably in a good position to just manage
resolv.conf yourself and having to add configuration and various options to
coax docker into producing a resolv.conf file you already know how to write
yourself seems wrong-headed.

Given this: I guess a simple "don't touch /etc/resolv.conf" option is my
current preference, or maybe something where you can pass the resolv.conf
to docker build/run/-d.


Reply to this email directly or view it on GitHubhttps://github.com//issues/1297#issuecomment-23164283
.

@FrozenCow

This comment has been minimized.

Copy link

@FrozenCow FrozenCow commented Nov 4, 2013

I like leaving all files controllable for the container and not specifically binding /etc/resolv.conf by default. That way we can do anything within the container. Ignoring /etc/resolv.conf (and others?) could be solved with something similar to .gitignore (global and container-specific). You can specify what files you want to ignore, which won't be shown in diff and won't be committed. With an ignore file we could also specify to ignore /var/log/** (for instance).

I'm kind of new to docker, so I might be missing some fundamental problems. I have been tinkering with lxc's utils, but there it also struck me that /etc/resolv.conf was being bound by default (and no options to turn this off). It is annoying when you want to install Ubuntu (and others) from scratch.

@clausa

This comment has been minimized.

Copy link

@clausa clausa commented Nov 5, 2013

I'm looking into using (Jenkins and) Docker to test puppet manifests and currently my "builds" fails due to puppet not being able to modify /etc/resolv.conf.

So +1 for any solution - or workaround - that would allow modifying /etc/resolv.conf :-)

@schnittchen

This comment has been minimized.

Copy link

@schnittchen schnittchen commented Nov 29, 2013

Like @clausa, I am running provisioning tests inside docker containers and these must be able to fiddle with /etc/resolv.conf. Any workaround making this possible is appreciated.

@schnittchen

This comment has been minimized.

Copy link

@schnittchen schnittchen commented Nov 29, 2013

@jpetazzo the symlink workaround no longer works, /var/lib/docker/containers/ is empty?

@ifischer

This comment has been minimized.

Copy link

@ifischer ifischer commented Feb 15, 2014

Any workaround for this when running Docker version 0.8.0, build cc3a8c8 ?

@parf

This comment has been minimized.

Copy link

@parf parf commented Feb 20, 2014

Advise:

  1. this files created as normal files on images deployment if there are no such files
    2.a user can specify their own files in docker run
    " docker run --copy "/etc/resolv.conf:/var/docker/XX/resolv.conf"
    2.b user can specify at least
    "namesever", "search" for resolv conf
    • this should alter This Lines Only !! in resolv.conf

^^ above should also apply for /etc/hosts

  • check how openvz does it, they divide /etc/hosts in two parts, one openvz controlled, another one is user controlled

PS:
our app uses "resolv.conf/search" a lot, internally we use short names everywhere.
so we can switch from dvp to production by changing "search" + /etc/hosts
some of this short names are name server controlled, some controlled by /etc/hosts

@crosbymichael crosbymichael removed this from the 0.7.1 milestone Mar 19, 2014
@crosbymichael crosbymichael removed their assignment Mar 19, 2014
@unclejack

This comment has been minimized.

Copy link
Contributor

@unclejack unclejack commented May 12, 2014

I'm closing this in favor of #2267 because that issue will also track the fix for this issue.

@unclejack unclejack closed this May 12, 2014
@christopherobin

This comment has been minimized.

Copy link

@christopherobin christopherobin commented May 18, 2015

Just a quick update, the implemented fixes do not fix that issue that is the resolvconf package for Debian, see the following apt output on debian jessie:

Setting up resolvconf (1.76.1) ...
ln: cannot remove '/etc/resolv.conf': Device or resource busy
dpkg: error processing package resolvconf (--configure):
 subprocess installed post-installation script returned error exit status 1

Which in turn prevents packages that depends on resolvconf from installing.
I could probably create a bunch of fake packages that provide those kind of broken packages, but I'd rather have a nicer way to make it work directly in docker.

jakirkham added a commit to jakirkham/docker_ubuntu_drmaa_conda that referenced this issue May 31, 2015
… subshell. This could actually be very hair because of two things. One Ubuntu prefers that one uses the `resolvconf` command installed via `apt-get`; however, this does install does not work nicely because of the next point. Two Docker mucks with `/etc/resolv.conf` in a way that Ubuntu might not like and in a way which we may not want to interfere ( moby/moby#1297 ). Regardless, we don't have much choice at present so we continue with the simplest solution.
jakirkham added a commit to jakirkham/docker_phusion_baseimage_drmaa_conda that referenced this issue Jun 3, 2015
… subshell. This could actually be very hair because of two things. One Ubuntu prefers that one uses the `resolvconf` command installed via `apt-get`; however, this does install does not work nicely because of the next point. Two Docker mucks with `/etc/resolv.conf` in a way that Ubuntu might not like and in a way which we may not want to interfere ( moby/moby#1297 ). Regardless, we don't have much choice at present so we continue with the simplest solution.
jakirkham added a commit to jakirkham/docker_phusion_baseimage_drmaa_conda that referenced this issue Jun 3, 2015
… subshell. This could actually be very hair because of two things. One Ubuntu prefers that one uses the `resolvconf` command installed via `apt-get`; however, this does install does not work nicely because of the next point. Two Docker mucks with `/etc/resolv.conf` in a way that Ubuntu might not like and in a way which we may not want to interfere ( moby/moby#1297 ). Regardless, we don't have much choice at present so we continue with the simplest solution.
@zuazo

This comment has been minimized.

Copy link

@zuazo zuazo commented Jun 25, 2015

@christopherobin, try the following workaround:

RUN :> /var/lib/resolvconf/linkified

This deceives the post-installation script. It worked for me in jessie.

@christopherobin

This comment has been minimized.

Copy link

@christopherobin christopherobin commented Jun 26, 2015

@zuazo I've ended up doing something similar:

RUN echo "resolvconf resolvconf/linkify-resolvconf boolean false" | debconf-set-selections
@LinBoLen

This comment has been minimized.

Copy link

@LinBoLen LinBoLen commented Jun 24, 2017

the problem.

specially happen when i switch to a difference wifi environment.

when use the default bridge network, the /etc/resolv.conf will mount host. (from above answer)
but when use docker network create foo-network, when container use foo-network network. the etc/resolv.conf will not bind.

default bridge

# Generated by dhcpcd from eth0.dhcp
# /etc/resolv.conf.head can replace this line
nameserver 192.168.65.1
# /etc/resolv.conf.tail can replace this line

created backend-tier

nameserver 127.0.0.11
options ndots:0

problem in gitlab ci

gitlab ci don't provide much alternative docker config. the linked service just use default bridge network

a lot of tears

ref: https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/2064

@LinBoLen

This comment has been minimized.

Copy link

@LinBoLen LinBoLen commented Jun 24, 2017

update

change host nameserver with a dns without pollution and also add gateway ip to a nameserver record may help(use command route -n in container, or docker inspect someofnetwork to find gateway)

because some of bad dns server will parse none exist domain to their ad ip.

really a lot of tears

JesusZapata added a commit to vauxoo-dev/docker-odoo-image that referenced this issue Nov 21, 2017
Using the follow solution
moby/moby#1297 (comment)
moylop260 added a commit to Vauxoo/docker-odoo-image that referenced this issue Nov 25, 2017
…r each version of python installed (#287)

* [FIX] odoo-shippable: Install the pylint_odoo and the depencies for each
version of python installed

* [FIX] odoo-shippable: Fix the error related with resolvconf
Using the follow solution
moby/moby#1297 (comment)

* [FIX] odoo-shippable: Fix the follow trouble "Package libffi was not
found"
@clippermadness

This comment has been minimized.

Copy link

@clippermadness clippermadness commented Mar 22, 2018

Hey folks, if you're like me and ended up here because you're trying to dockerize some old busted software that insists on installing resolvconf on Ubuntu 12.04, you can workaround it by configuring resolvconf to skip the post install step where it link-ifies /etc/resolv/conf:

RUN apt-get -y install debconf-utils && echo resolvconf resolvconf/linkify-resolvconf boolean false | debconf-set-selections && apt-get -y install resolvconf

Waldz added a commit to mysteriumnetwork/node that referenced this issue Jul 3, 2018
see moby/moby#1297

Signed-off-by: Waldz <valdas@mysterium.network>
Waldz added a commit to Waldz/node that referenced this issue Jul 3, 2018
see moby/moby#1297

Signed-off-by: Waldz <valdas@mysterium.network>
Waldz added a commit to Waldz/node that referenced this issue Jul 4, 2018
see moby/moby#1297

Signed-off-by: Waldz <valdas@mysterium.network>
Waldz added a commit to Waldz/node that referenced this issue Jul 5, 2018
see moby/moby#1297

Signed-off-by: Waldz <valdas@mysterium.network>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.