Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add volumes for docker driver #62

Closed
cbednarski opened this issue Sep 19, 2015 · 18 comments
Closed

Add volumes for docker driver #62

cbednarski opened this issue Sep 19, 2015 · 18 comments

Comments

@cbednarski
Copy link
Contributor

@cbednarski cbednarski commented Sep 19, 2015

Similar in theme to #60 and #61, we want to share data between tasks in a group. We will need to mount the task group allocation directory as a docker volume so we can share data between these processes.

@cbednarski cbednarski assigned cbednarski and unassigned cbednarski Sep 24, 2015
@zrml
Copy link

@zrml zrml commented Sep 30, 2015

+1
Fundamental!
Thanks

@crhuber
Copy link

@crhuber crhuber commented Oct 12, 2015

+1

@sshi100
Copy link

@sshi100 sshi100 commented Oct 22, 2015

+1

@dadgar
Copy link
Contributor

@dadgar dadgar commented Oct 22, 2015

This PR closes this #290

@dadgar dadgar closed this Oct 22, 2015
@calvn
Copy link
Member

@calvn calvn commented Dec 22, 2015

Is the doc updated with this? I don't see anything on https://nomadproject.io/docs/drivers/docker.html regarding volumes.

@diptanu
Copy link
Collaborator

@diptanu diptanu commented Dec 22, 2015

@cleung2010 So this issue was about tasks in a group sharing an ephemeral volume. If you're looking for mounting host volumes in docker containers, that would land in the future.

@c4milo
Copy link
Contributor

@c4milo c4milo commented Dec 22, 2015

@diptanu do you mean we cannot mount parts of the host's filesystem into Docker containers right now?

@calvn
Copy link
Member

@calvn calvn commented Dec 22, 2015

@c4milo I think that's the case, don't see that in the docs or source. I believe @diptanu's response confirms it.

@c4milo
Copy link
Contributor

@c4milo c4milo commented Dec 22, 2015

@cleung2010 I believe you can pass -v arguments to the docker command, but I haven't tested it to be really sure. That's why I'm asking, I was under the impression this was possible today.

@c4milo
Copy link
Contributor

@c4milo c4milo commented Dec 22, 2015

ah nevermind, it is intended for the actual binary run with the container. So, it is not possible.

@diptanu
Copy link
Collaborator

@diptanu diptanu commented Dec 23, 2015

Ok so we currently create an allocation directory on the host and mount it on every container, so every container gets a host volume for writing data. But it is currently not possible to mount an arbitrary path on the host on a container.

This involves allowing operators to configure which paths on the host is allowed for mounting and which tasks can mount which paths on the host, etc. We want the design to be right so we haven't done anything on this yet. But this is something we understand is very important, so I think it would be nice to just down a bunch of use cases that would need us to mount host volumes.

@c4milo
Copy link
Contributor

@c4milo c4milo commented Dec 23, 2015

@diptanu, so my current use case is being able to access the host's Consul agent from containers. I was thinking in sharing the directory where unix sockets are being created so containers can use them. Unfortunately, the current allocation directory implementation does not seem to help with this specific use case as it purges the allocation directory shared among the containers.

@diptanu
Copy link
Collaborator

@diptanu diptanu commented Dec 24, 2015

@c4milo Yeah we have been wondering how Tasks which have network isolation could talk to Consul, I guess that would be a good way to implement this.

@erich-comsIO
Copy link

@erich-comsIO erich-comsIO commented Dec 25, 2015

I've been using cephfs to persist and share volumes across containers using host volume mounts when starting docker. For example, I bring up containers and mount users' home directories, and that's very important for my application.

Should I be mounting cephfs volumes within the containers themselves? That would (I think) require the containers run in privileged mode which would seem otherwise unnecessary.

@ketzacoatl
Copy link
Contributor

@ketzacoatl ketzacoatl commented Dec 25, 2015

I have a simple use case I'm sure lots of other people do: I'd like to run a docker task which mounts a volume as a path from the host. For example, in my case, a configuration management or logging agent would need access to the underlying host filesystem.

I have a lot of respect for Hashicorp and the projects you've collectively put forth, and I am often very grateful for the patience and care put into the abstractions, interfaces, etc, that are put to use.. in many respects, that is what sets Hashicorp apart from the others.. but this situation here seems counter-intuitive.

Please confirm, is our workaround for the moment to enable to raw_exec driver so we can run docker run .... -v ... ....? Seems silly to do that instead of using the docker driver directly, and only because there's no easy way to tell a task "add this filesystem path as a volume".

@zapman449
Copy link

@zapman449 zapman449 commented Apr 18, 2016

Another use case: Volumes are frequently how secure details like Certificates are injected into web servers. Building the cert into the docker image is terrible. And a plethora of little loader scrips as the entry point into said images is an anti-pattern and should be avoided where possible. (loader script to go retrieve a cert from )

@dvusboy
Copy link
Contributor

@dvusboy dvusboy commented Apr 18, 2016

Yet one more: There are some system services that I'd like to use that would preferably be launched as containers. Some of them require access to the engine socket. The typical way to handle this with --volume=/var/run/docker.sock:/var/run/docker.sock. There is currently no way to do this with nomad. Neither exec nor raw_exec is acceptable alternative: Do you run the container attached, for example?

@ketzacoatl
Copy link
Contributor

@ketzacoatl ketzacoatl commented Apr 19, 2016

In most cases, it's hard to say raw_exec would not be sufficient, though with docker it means you need to create a wrapper script to do it correctly. The general flow in the wrapper is to:
a) remove the named container if one exists (the script should || true or similar to avoid exiting if the named container does not exist)
b) create a new named container
c) start that named container in daemon mode, then
d) detached, run docker logs with "follow" (similar to tail -f), the script should exist if that docker logs exits (which means the container failed/stopped)

This follows the recommendations from docker upstream, see moby/moby#6791 for more details on that.

schmichael pushed a commit that referenced this issue Jan 30, 2020
Sync oss for 0.7.1 RC1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.