New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for sharing unix sockets #483

Open
TheCodeNinja opened this Issue Aug 31, 2016 · 99 comments

Comments

Projects
None yet
@TheCodeNinja

TheCodeNinja commented Aug 31, 2016

Expected behavior

When mounting a directory containing unix sockets the sockets should function the same as they do on a Linux host.

Actual behavior

The socket is 'there', but non-functional.

Information

After reading several forum threads, it appears that there is a workaround with socat over TCP, but this is rather slow.

The documentation has this to say: 'Socket files and named pipes only transmit between containers and between OS X processes -- no transmission across the hypervisor is supported, yet'
Hopefully this is a planned feature already, but I did not see any existing issues open in this tracker for this particular issue, although it relates to #410 which asks specifically for SSH_AUTH_SOCK to be supported.

Host OS: Mac OSX 10.10.5

Steps to reproduce the behavior

  1. mount a directory containing unix sockets like so: '-v "/directorywithsockets:/otherdirectory"'
  2. attempt to send data to/from the host/container via the socket
@dsheets

This comment has been minimized.

Contributor

dsheets commented Sep 1, 2016

This is on the roadmap.

@udondan

This comment has been minimized.

udondan commented Sep 19, 2016

Interestingly, even a socket created in the container is non-functional if it was created on a mounted volume of the host.

@jippi

This comment has been minimized.

jippi commented Oct 15, 2016

Any ETA on this? :)

Currently it's blocking hashicorp/nomad#1091 and other projects :)

@dsheets

This comment has been minimized.

Contributor

dsheets commented Oct 15, 2016

This is currently scheduled for resolution in November. Sorry for the delay.

@jippi

This comment has been minimized.

jippi commented Oct 15, 2016

@dsheets okay, will it be available in beta builds beforehand?

@dsheets

This comment has been minimized.

Contributor

dsheets commented Oct 15, 2016

Our goal is to ship it in beta builds in November.

@jippi

This comment has been minimized.

jippi commented Oct 15, 2016

amazing @dsheets - if you need any testing or otherwise beforehand, I'm willing to help out!

@Druotic

This comment has been minimized.

Druotic commented Oct 28, 2016

Our goal is to ship it in beta builds in November.

@dsheets That statement makes me think the work for this is already underway. However, the status label is still "1-acknowledged". Is this currently being worked on, or still on the todo list?

@WyseNynja

This comment has been minimized.

WyseNynja commented Nov 9, 2016

We are in November now :) Any guess when this will be in Beta? This feature blocks a lot for us.

@Druotic

This comment has been minimized.

Druotic commented Nov 16, 2016

Any updates? Even a mention that this feature has been abandoned would be better than nothing.

@dsheets

This comment has been minimized.

Contributor

dsheets commented Nov 16, 2016

The feature has not been abandoned. It is still on the roadmap. Thanks for your patience.

@WyseNynja

This comment has been minimized.

WyseNynja commented Nov 30, 2016

It is the last day of November. Any update on when this will be in the beta @dsheets?

@dsheets

This comment has been minimized.

Contributor

dsheets commented Dec 1, 2016

Work has begun but is currently delayed behind performance work. Sorry about that. Stay subscribed to get updates. Thanks for your patience!

@gugahoi

This comment has been minimized.

gugahoi commented Dec 22, 2016

Any updates on this?

@WyseNynja

This comment has been minimized.

WyseNynja commented Dec 28, 2016

I've started using https://github.com/avsm/docker-ssh-agent-forward to work around this issue

@WyseNynja

This comment has been minimized.

WyseNynja commented Jan 19, 2017

Any updates @dsheets ? Is there an issue tracking the performance work? If that is going to block for much longer, I am planning to spend some time cleaning up https://github.com/avsm/docker-ssh-agent-forward but if docker will have proper support soon I won't bother.

@KFoxder

This comment has been minimized.

KFoxder commented Jan 19, 2017

+1

@dsheets

This comment has been minimized.

Contributor

dsheets commented Jan 20, 2017

Support for this is planned and has been started but is delayed indefinitely behind other work. @WyseNynja I would recommend cleaning up avsm/docker-ssh-agent-forward in the meantime.

@tomgeorge

This comment has been minimized.

tomgeorge commented May 11, 2018

So... I found something interesting in this for my use case (sharing docker.sock for local development, not really concerned about the security of the socket), but I have no idea what the implications are of this.

On the OS X host, /var/run/docker.sock is owned by root:daemon, and when you run a container with -v /var/run/docker.sock:/var/run/docker.sock, docker.sock is owned by root:root.

I was looking into trying something with socat to forward this into the container (which I don't fully understand, either), and just for grins I did the following

docker run -v /var/run/docker.sock:/var/run/docker.sock -it jpetazzo/dind /bin/bash
# docker.sock is owned by root:root at this point
chown root:daemon /var/run/docker.sock
usermod -Ums /bin/bash -G daemon tom
su tom
docker run hello-world

That seemed to work. Running docker in OS X still worked after exiting the container, and on subsequent docker runs the permission change persisted until I restarted docker for mac.

I don't really know why this works, or if I'm going to hose some part of my system, but it worked for me, I can now run docker on the host and in a container.

Docker for mac Version 18.03.1-ce-mac65 (24312)
OS X 10.13.4

cjh1 added a commit to cjh1/tomviz that referenced this issue May 30, 2018

Use file based progress on MacOS
Sharing of local socket is not currently supported on MacOS, see
docker/for-mac#483 for details.

huntc added a commit to landlord/landlord that referenced this issue Jun 1, 2018

Support docker and Unix Domain socket mounts
Unfortunately, on OS X, subject to a problem here: docker/for-mac#483
@4ntoine

This comment has been minimized.

4ntoine commented Jul 16, 2018

So are Unix sockets supported by docker eventually (host: macOs, container: linux)?

@Alan-R

This comment has been minimized.

Alan-R commented Jul 17, 2018

You can do some things (described above) which allow the two to inter-operate if all you need is data connectivity. But, if you need full Linux sockets APIs (like I described earlier) - that is unlikely to ever happen.

@trinitronx

This comment has been minimized.

trinitronx commented Jul 17, 2018

It seems that the most prevalent use case for this feature is for developers to develop and test containers locally on a single-user MacOS laptop. In this case, the connectivity is the main feature needed while the extra security & user identification provided natively on Linux are probably less useful. Most common examples being:

  • Mounting Docker socket for launching other containers & interacting with Docker daemon from a container
  • Mounting SSH Agent Socket for SSH connections from a container

Correct me if I'm wrong, but I've never heard of any company using a production environment running containers hosted on macOS Server. Therefore security features are lower priority given that the developer will usually trust themselves, or at least take responsibility if they do things that accidentally mess up the host system somehow. In this case, the "customer" (e.g. developers) for the feature really just wants working sockets through host volume mounts to work just like on Linux. It could be that some use cases for using sockets that require these extra features will not work, but at least basic connectivity is satisfied.

@Alan-R

This comment has been minimized.

Alan-R commented Jul 17, 2018

I agree with @trinitronx about why people use this and why the security features don't
matter to most people - unless like me, they're testing and developing security sensitive software which needs this feature.

Given that the Docker OS environment and the MacOS environment share nothing but the contents of regular files, and have no pid space in common, and even have different maximum values for process ids, sharing that kind of security information would require a very radical and costly rearchitecture of the MacOS docker implementation, or a switch of MacOS to become Linux-based. IMHO, neither seems very likely - exactly for the reasons @trinitronx stated. Although the former is somewhat more likely than the latter ;-).

@lox

This comment has been minimized.

lox commented Jul 17, 2018

One of the primary reasons we have Docker for Mac is to test and develop Docker workflows that will run on Linux environments. Without parity around how sockets are handled, it's very difficult to work with any applications that mount in an ssh-agent, or the myriad of other use cases folks above have described. The alternative is developing custom alternate implementations for when the host is macOS, which kind of devalues some of the core benefits of docker and docker-compose based workflows.

@zachmullen

This comment has been minimized.

zachmullen commented Jul 31, 2018

mounting socket or dir won't work for mac, because unix socket is kind of in os memory, but docker and mac is not the same kernel. you'll get the socket on the other side, but it's not "connected" because logically speaking those sockets are in two different machines.

How is that fundamentally different from the stdin, stdout, and stderr pipes? I'd like to be able to use named pipes, which should in theory work almost the exact same as the process standard pipes in a hypervisor environment. Can someone explain to me why docker on MacOS supports stdout/stderr/stdin, but not named pipes? Is there some critical difference I'm missing?

@kler

This comment has been minimized.

kler commented Aug 1, 2018

@docker-for-desktop-robot

This comment has been minimized.

Collaborator

docker-for-desktop-robot commented Oct 30, 2018

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

@href

This comment has been minimized.

href commented Oct 30, 2018

/remove-lifecycle stale

@bryanhuntesl

This comment has been minimized.

bryanhuntesl commented Nov 21, 2018

http://collabnix.com/how-docker-for-mac-works-under-the-hood/

Docker for Mac does not use docker-machine to provision its VM. The Docker Engine API is exposed on a socket available to the Mac host at /var/run/docker.sock. This is the default location Docker and Docker Compose clients use to connect to the Docker daemon, so you to use docker and docker-compose CLI commands on your Mac.

So.. if it can expose that socket, how come it can't handle SSH_AUTH_SOCK ?

@djpadz

This comment has been minimized.

djpadz commented Nov 21, 2018

@bryanhuntesl said:

So.. if it can expose that socket, how come it can't handle SSH_AUTH_SOCK ?

Because the docker socket is being exposed by a daemon running under macOS. The challenge with SSH_AUTH_SOCK is that it needs to be exposed inside of a running container, which is actually running inside of a VM. So, the socket would be caught by the VM's kernel, instead of the macOS kernel. It's like trying to mount a socket from one machine to another via NFS. The socket file exists, but its metadata doesn't make any sense to the kernel on the remote machine.

@WyseNynja

This comment has been minimized.

WyseNynja commented Nov 21, 2018

@tamsky

This comment has been minimized.

tamsky commented Nov 21, 2018

@WyseNynja writes

It looks like docker 2 added “docker build —ssh AUTHSOCK”

I couldn't find this in any release notes. Can you share a link to this new feature?

@tamsky

This comment has been minimized.

tamsky commented Nov 21, 2018

I think this is it... I had been looking at docker/for-mac release notes.

docker/cli#1014

@WyseNynja

This comment has been minimized.

WyseNynja commented Nov 22, 2018

docker/cli#1014 is something else (though it may also be useful for the people in this thread).

I was talking about https://medium.com/@tonistiigi/build-secrets-and-ssh-forwarding-in-docker-18-09-ae8161d066

@bryanhuntesl

This comment has been minimized.

bryanhuntesl commented Nov 22, 2018

@bryanhuntesl said:

So.. if it can expose that socket, how come it can't handle SSH_AUTH_SOCK ?

Because the docker socket is being exposed by a daemon running under macOS. The challenge with SSH_AUTH_SOCK is that it needs to be exposed inside of a running container, which is actually running inside of a VM. So, the socket would be caught by the VM's kernel, instead of the macOS kernel. It's like trying to mount a socket from one machine to another via NFS. The socket file exists, but its metadata doesn't make any sense to the kernel on the remote machine.

Thanks for clarifying.

@jeantil

This comment has been minimized.

jeantil commented Nov 22, 2018

docker/cli#1014 is something else (though it may also be useful for the people in this thread).

Is it possible to connect to the docker4mac host vm using this ? (export DOCKER_HOST=ssh://localhost:????), then it should be possible to use ssh agent forwarding to share the osx based agent with a container ?

I am very very very interested in a solution for this. I have been hardening my security practices for the last couple years and this is one of the last issue with my setup.

I cannot copy my ssh private key nor share it through a volume (I use gpg-agent with ssh-agent mode and the corresponding gpg private key is safely stored on a write-only yubikey) therefore most workarounds to share my ssh credentials with docker containers are unapplicable for me.

Since it is also impossible to map a USB device to a container (as far as my research indicates) I was forced to create a less secure ssh key to be used in containers which is not so nice and means I have abandoned docker for some use cases (managing my ansible environment for instance).

@adamnovak

This comment has been minimized.

adamnovak commented Dec 12, 2018

Is this the right issue for tracking cross-hypervisor FIFO support? Or is this issue only for Unix sockets and their special features, and plain text streams need their own issue?

It seems like plain streams have a relatively successful workaround of "just use a network connection instead". Is that going to be (or has it been) rolled in as a workaround in Docker for Mac itself, or should I spend time implementing it in my project?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment