New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker volumes w remote daemon #4023

Closed
bradfitz opened this Issue Feb 9, 2014 · 134 comments

Comments

Projects
None yet
@bradfitz
Copy link

bradfitz commented Feb 9, 2014

This seems like a tricky one, but I noticed that I can't use "docker run -v ..." with Docker 0.8 on OS X with boot2docker.

Also, I get no error message. My docker container just fails to run properly because paths are missing. At minimum, docker run should fail with an "unsupported on OS X" error message.

But ideally, since boot2docker in VirtualBox is on a different (virtual) machine than the OS X host and we can't simply bind mount between them, perhaps a FUSE filesystem can be mounted in the Linux VM that forwards back to the OS X host.

@kovyrin

This comment has been minimized.

Copy link

kovyrin commented Feb 9, 2014

Spent some time debugging this issue as well. I don't think it is a os x specific problem. Every time one uses docker via a remote tcp connection to the server, there is a huge change -v means something really different when what user expects from it. Some warning message about it could probably be displayed when docker_host uses tcp.

@narent

This comment has been minimized.

Copy link

narent commented Feb 11, 2014

This one tripped me up too. Agree with Koviryn. A simple message that the the client is talking over tcp and cannot mount the volume from the client's machine would be very helpful in debugging.

@vieux

This comment has been minimized.

Copy link
Collaborator

vieux commented Feb 11, 2014

You can still use tcp when client and daemon are on the same machine.
So, if we can find something else, the connection type shouldn't be the
thing that trigger the warning
On Feb 11, 2014 3:44 AM, "Narendra Tumkur" notifications@github.com wrote:

This one tripped me up too. Agree with Koviryn. A simple message that the
the client is talking over tcp and cannot mount the volume from the
client's machine would be very helpful in debugging.

Reply to this email directly or view it on GitHubhttps://github.com//issues/4023#issuecomment-34746912
.

russroy added a commit to russroy/docker that referenced this issue Feb 22, 2014

Update working_with_volumes.rst
Add to list of known issues:

docker run -v /host/path:/container/path doesn't work on Mac OS X moby#4023
moby#4023
@russroy

This comment has been minimized.

Copy link

russroy commented Feb 23, 2014

This issue is a major impediment to anyone trying to do Packer/Docker on OS X w/ boot2docker and the like.

@bradfitz

This comment has been minimized.

Copy link
Author

bradfitz commented Mar 20, 2014

Happy Birthday, Docker!

For your birthday, I propose we try to fix this. Let's make a FUSE filesystem. People here at the Docker Birthday Party don't think it's crazy.

The FUSE filesystem (the server) will run on Linux only (in/under dockerd) and then the client will just be the docker client (Mac/Windows/Linux/whatever). Performance should be great or good with low-ish latency (localhost or local network) and probably acceptable even with Internetish latency for just writing out the result of commands (e.g. getting the apk file out of an Android app build done in a container).

WIre protocol will not be NFS, SMB, or something like that. We'll just do a pretty 1:1 mapping of the FUSE protocol to a slightly simpler wire format... fixed packet headers of (request/response ID, body length, packet type id), and then $body_length of body bytes. Multiple requests can be in flight at once, just like FUSE.

We'll probably want to then vendor the bazil FUSE libraries.

@shykes

This comment has been minimized.

Copy link
Collaborator

shykes commented Mar 20, 2014

Don't you guys prefer doing SMB? It would have the extra advantage of working on every Mac without requiring installing macfuse.

On Thu, Mar 20, 2014 at 2:36 PM, Brad Fitzpatrick
notifications@github.com wrote:

Happy Birthday, Docker!
For your birthday, I propose we try to fix this. Let's make a FUSE filesystem. People here at the Docker Birthday Party don't think it's crazy.
The FUSE filesystem (the server) will run on Linux only (in/under dockerd) and then the client will just be the docker client (Mac/Windows/Linux/whatever). Performance should be great or good with low-ish latency (localhost or local network) and probably acceptable even with Internetish latency for just writing out the result of commands (e.g. getting the apk file out of an Android app build done in a container).
WIre protocol will not be NFS, SMB, or something like that. We'll just do a pretty 1:1 mapping of the FUSE protocol to a slightly simpler wire format... fixed packet headers of (request/response ID, body length, packet type id), and then $body_length of body bytes. Multiple requests can be in flight at once, just like FUSE.

We'll probably want to then vendor the bazil FUSE libraries.

Reply to this email directly or view it on GitHub:
#4023 (comment)

@bradfitz

This comment has been minimized.

Copy link
Author

bradfitz commented Mar 20, 2014

No, I absolutely do not. :-) Macs and Windows won't need to install FUSE anything. FUSE is only in dockerd on Linux. The client (the Go docker command) doesn't speak FUSE or SMB. It speaks a custom wire protocol that we define, that maps closely to what our FUSE server needs, but is it not FUSE itself. No kernel extensions will be needed on the client machine running the docker command.

@shykes

This comment has been minimized.

Copy link
Collaborator

shykes commented Mar 20, 2014

Oh wow! Awesome!

On Thu, Mar 20, 2014 at 2:57 PM, Brad Fitzpatrick
notifications@github.com wrote:

No, I absolutely do not. :-) Macs and Windows won't need to install FUSE anything. FUSE is only in dockerd on Linux. The client (the Go docker command) doesn't speak FUSE or SMB. It speaks a custom wire protocol that we define, that maps closely to what our FUSE server needs, but is it not FUSE itself. No kernel extensions will be needed on the client machine running the docker command.

Reply to this email directly or view it on GitHub:
#4023 (comment)

@thaJeztah

This comment has been minimized.

Copy link
Member

thaJeztah commented Mar 21, 2014

+1 sounds great; if I understand correctly, that would also allow me to install the docker client on my mac, working with a remote (non-VM) docker host and mount my local system as volume?

Thinking of this for central in-house docker host for development and using local workstation files as volumes without having to ssh/scp them to the docker-host

@SvenDowideit

This comment has been minimized.

Copy link
Contributor

SvenDowideit commented Mar 21, 2014

So docker run -v /host:/container... would need to leave something running on the client side?

I'm presuming that you're building this now ? :) shame we don't have a windows build of the docker client we can test that on too.

@SvenDowideit

This comment has been minimized.

Copy link
Contributor

SvenDowideit commented Mar 21, 2014

@shykes have you talked to @bradfitz about the approach we discussed? It aught to be less code, and allow connections to remote FS's that are not on the client host as well (but does require a file server daemon of some kind)

ie, if @bradfitz 's FUSE code is actually a volume-driver (and that we can have more than one volume-driver at one time), then we get lots of magic.

@SvenDowideit SvenDowideit changed the title docker run -v /host/path:/container/path doesn't work on Mac OS X docker run -v /host/path:/container/path doesn't work on remote Docker daemons Mar 21, 2014

@lazywei

This comment has been minimized.

Copy link

lazywei commented Apr 8, 2014

+1 for this. Any progress on this one?

@aidanhs

This comment has been minimized.

Copy link
Contributor

aidanhs commented Apr 11, 2014

I'm interested in where I can look at the actual progress of this (to comment, try out, hack on etc).
Looks like it might be this one: https://github.com/bradfitz/docker/commits/fuse ? Are comments/issues/PRs acceptable, or is the design too raw still?

One thing I'm curious about: "WIre protocol will not be NFS, SMB, or something like that."
Assuming that stripped down versions of NFS etc are still too much, would a basis on sshfs be at all interesting? Are there going to be tangible benefits to this custom protocol? It still has to transfer files and permissions, which is (approximately) what sshfs does.

@philips

This comment has been minimized.

Copy link
Contributor

philips commented Apr 11, 2014

Thinking about this again we could use go9p and used 9p mount on the Docker side.

@pnasrat

This comment has been minimized.

Copy link
Contributor

pnasrat commented Apr 11, 2014

Is go9p actively maintained?

https://code.google.com/p/go9p/source/list

Doesn't show any recent changes, there is an issue pointing at

https://code.google.com/p/goplan9/source/list

Which does have some more recent changes

On 11 April 2014 12:34, Brandon Philips notifications@github.com wrote:

Thinking about this again we could use go9p and used 9p mount on the
Docker side.

Reply to this email directly or view it on GitHubhttps://github.com//issues/4023#issuecomment-40223467
.

@bradfitz

This comment has been minimized.

Copy link
Author

bradfitz commented Apr 11, 2014

Using the sshfs protocol might be reasonable. I haven't looked at it.

I was going to switch the docker fuse stuff to use protobufs instead with the existing framing format (request ID & length). We could drop the outer "type id" from the frame and just specify that the requests and replies are always of one specific protobuf message type, with a bunch of optional sub-messages.

Sorry that I haven't gotten back to this after the hackathon. I've been closing up my Go 1.3 bugs, but that's largely done. I'll try to chip away at this. Anybody around the bay area want to meet up and co-hack on it sometime?

@pnasrat

This comment has been minimized.

Copy link
Contributor

pnasrat commented Apr 11, 2014

Are you going to be at Gophercon? If so I could meet up then and experiment
with some ideas.

Paul

On 11 April 2014 12:59, Brad Fitzpatrick notifications@github.com wrote:

Using the sshfs protocol might be reasonable. I haven't looked at it.

I was going to switch the docker fuse stuff to use protobufs instead with
the existing framing format (request ID & length). We could drop the outer
"type id" from the frame and just specify that the requests and replies are
always of one specific protobuf message type, with a bunch of optional
sub-messages.

Sorry that I haven't gotten back to this after the hackathon. I've been
closing up my Go 1.3 bugs, but that's largely done. I'll try to chip away
at this. Anybody around the bay area want to meet up and co-hack on it
sometime?

Reply to this email directly or view it on GitHubhttps://github.com//issues/4023#issuecomment-40225834
.

@bradfitz

This comment has been minimized.

Copy link
Author

bradfitz commented Apr 11, 2014

I will be, yes.

@vvisigoth

This comment has been minimized.

Copy link

vvisigoth commented Apr 15, 2014

Sorry, I'm late to the party on this. Is there any word on how to work around

docker run -v

not working with boot2docker on OS X?

@xeor

This comment has been minimized.

Copy link

xeor commented Apr 20, 2014

Is there a plan here? https://github.com/bradfitz/docker/commits/fuse hasn’t been active for a month now.

I can see that this might be more complicated to implement than you will think at first, but I think it is a blocker for many people out there.
Probably most of us are using OSX for their developer workstation, and needing to be inside the container to do instant code changes, or to view logs is not fun..

As a workaround, I am looking into mapping a folder on my boot2docker vm to my mac but it isn't a good solution..

@bradfitz

This comment has been minimized.

Copy link
Author

bradfitz commented Apr 20, 2014

Sorry, this is all very straight-forward but I just haven't made time to finish it. I'm thinking after Gophercon will be less chaotic.

@xeor

This comment has been minimized.

Copy link

xeor commented Apr 20, 2014

@bradfitz Thanks for the status update, I think many of us will be pleased just to hear that the problem is not forgotten and you think it is straight forward!

For those people to lazy to google, Gophercon ends 26th of April (NOT meaning that this will be fixed by the 27th(!)).. Have fun

@russroy

This comment has been minimized.

Copy link

russroy commented Apr 28, 2014

+1

@duro

This comment has been minimized.

Copy link

duro commented Dec 5, 2014

Is there any future for this? And will it for THE LOVE OF GOD solve the problem with VBox Shared Folders not supporting inotify?

@thaJeztah

This comment has been minimized.

Copy link
Member

thaJeztah commented Dec 5, 2014

This issue in particular is why I wanted to add protocol support to the -v command, so one could do -v fuse:// or -v ceph://

I'd rather have the protocol (and other settings) "abstracted" away from the run command. So first define a volume (and the protocol to use), then use it for a container (and refer to the volume by its name/id). This makes the volumes more manageable and flexible. The storage backend / protocol used for a volume should not be important to the run command; just "use this storage, and mount it".

Also see my comment on the other proposal #9250 (comment)

@cpuguy83

This comment has been minimized.

Copy link
Contributor

cpuguy83 commented Dec 5, 2014

I totally get that.

@sminnee

This comment has been minimized.

Copy link

sminnee commented Feb 10, 2015

@vieux if you had a chance, it would be great to get an update on whether the FUSE work was still something you planned to work on in the near future

@jessfraz jessfraz changed the title docker run -v /host/path:/container/path doesn't work on remote Docker daemons remote Docker volumes: docker run -v to remote daemon Feb 26, 2015

@jessfraz jessfraz changed the title remote Docker volumes: docker run -v to remote daemon docker volumes w remote daemon Feb 26, 2015

yuval-k pushed a commit to yuval-k/compose that referenced this issue Apr 10, 2015

Add warning for mapping local volumes
Mapping local volumes is not currently supported in boot2docker

boot2docker/boot2docker#413
moby/moby#4023

Signed-off-by: Saul Shanabrook <s.shanabrook@gmail.com>

Signed-off-by: Yuval Kohavi <yuval.kohavi@gmail.com>
@justjacksonn

This comment has been minimized.

Copy link

justjacksonn commented Jun 9, 2015

So this is a few months later and I still don't see/read anything related to whether or not docker on Mac can use the -v option to share host volumes with docker volumes... is there anything other than the sshfs solution? I've tried that and it is not working for me either. Can't write to the mapped folder despite use defer_permissions and other options. It's not easy to understand how to use that process and it seems silly that we have to run an external app and ssh to boot2docker to get host volumes working. Hopefully I have an old boot2docker build and it's now resolved?

@fyddaben

This comment has been minimized.

Copy link

fyddaben commented Sep 10, 2015

sshfs is so slow .

@cpuguy83

This comment has been minimized.

Copy link
Contributor

cpuguy83 commented Sep 10, 2015

Docker 1.8 supports volume plugins which you can use to implement remote volumes (of which there are a few implementations already).
This doesn't generally give you any client-side magic but is still extremely useful.
For example docker run -v <some name>:/foo --volume-driver=smb.

Docker 1.9 will allow you to do docker run -v $(docker volume create --name <some name> --driver smb --opt <some opt> --opt <some other opt>):/foo

@fyddaben

This comment has been minimized.

Copy link

fyddaben commented Sep 10, 2015

Thansk @homerjam sshfs is very useful. my code is on my container, and i sshfs my container on my OSX. so thank you .thank you thank you
my command is

sshfs -p 2023 -o allow_other work@$(boot2docker ip):/home/work/data/vi/  /Volumes/01/vc

so i can share my file(in my container) on my OSX .

@docteurklein

This comment has been minimized.

Copy link
Contributor

docteurklein commented Sep 16, 2015

@docteurklein

This comment has been minimized.

Copy link
Contributor

docteurklein commented Sep 16, 2015

Talking about that: what would it look like to use docker-machine's ssh config with this docker-volume plugin?

@homerjam

This comment has been minimized.

Copy link

homerjam commented Feb 23, 2016

This tool looks useful https://github.com/codekitchen/dinghy it appears to use NFS under the hood to work around the shared folders issue

@icecrime

This comment has been minimized.

Copy link
Contributor

icecrime commented Sep 10, 2016

@bradfitz Do you consider this one solved by Docker 4 Mac?

@thaJeztah

This comment has been minimized.

Copy link
Member

thaJeztah commented Sep 19, 2016

closing because there's no activity, but happy to reopen if you think there's still something to be done

@thaJeztah thaJeztah closed this Sep 19, 2016

@bradfitz

This comment has been minimized.

Copy link
Author

bradfitz commented Sep 20, 2016

Yeah, I'm happy enough. xhyve is fun.

@thaJeztah

This comment has been minimized.

Copy link
Member

thaJeztah commented Sep 20, 2016

Thanks @bradfitz

@irjudson

This comment has been minimized.

Copy link

irjudson commented Sep 28, 2016

The solutions for docker for windows have not resolved this family of issues, most of which were closed as a duplicate of this one.

What's the current thinking or solution for volume sharing not working on docker for windows?

@thaJeztah

This comment has been minimized.

Copy link
Member

thaJeztah commented Sep 28, 2016

@irjudson if there are still issues, the best way is to check the docker for windows issue tracker (https://github.com/docker/for-win/issues). Implementing / fixing that is specific for Docker for Windows, and cannot be resolved in this repository, so for that reason it's not needed to keep this issue open here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment