Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for sharing unix sockets #483

Closed
BouncyLlama opened this issue Aug 31, 2016 · 132 comments
Closed

Support for sharing unix sockets #483

BouncyLlama opened this issue Aug 31, 2016 · 132 comments

Comments

@BouncyLlama
Copy link

@BouncyLlama BouncyLlama commented Aug 31, 2016

Expected behavior

When mounting a directory containing unix sockets the sockets should function the same as they do on a Linux host.

Actual behavior

The socket is 'there', but non-functional.

Information

After reading several forum threads, it appears that there is a workaround with socat over TCP, but this is rather slow.

The documentation has this to say: 'Socket files and named pipes only transmit between containers and between OS X processes -- no transmission across the hypervisor is supported, yet'
Hopefully this is a planned feature already, but I did not see any existing issues open in this tracker for this particular issue, although it relates to #410 which asks specifically for SSH_AUTH_SOCK to be supported.

Host OS: Mac OSX 10.10.5

Steps to reproduce the behavior

  1. mount a directory containing unix sockets like so: '-v "/directorywithsockets:/otherdirectory"'
  2. attempt to send data to/from the host/container via the socket
@dsheets
Copy link
Contributor

@dsheets dsheets commented Sep 1, 2016

This is on the roadmap.

@udondan
Copy link

@udondan udondan commented Sep 19, 2016

Interestingly, even a socket created in the container is non-functional if it was created on a mounted volume of the host.

@jippi
Copy link

@jippi jippi commented Oct 15, 2016

Any ETA on this? :)

Currently it's blocking hashicorp/nomad#1091 and other projects :)

@dsheets
Copy link
Contributor

@dsheets dsheets commented Oct 15, 2016

This is currently scheduled for resolution in November. Sorry for the delay.

@jippi
Copy link

@jippi jippi commented Oct 15, 2016

@dsheets okay, will it be available in beta builds beforehand?

@dsheets
Copy link
Contributor

@dsheets dsheets commented Oct 15, 2016

Our goal is to ship it in beta builds in November.

@jippi
Copy link

@jippi jippi commented Oct 15, 2016

amazing @dsheets - if you need any testing or otherwise beforehand, I'm willing to help out!

@Druotic
Copy link

@Druotic Druotic commented Oct 28, 2016

Our goal is to ship it in beta builds in November.

@dsheets That statement makes me think the work for this is already underway. However, the status label is still "1-acknowledged". Is this currently being worked on, or still on the todo list?

@WyseNynja
Copy link

@WyseNynja WyseNynja commented Nov 9, 2016

We are in November now :) Any guess when this will be in Beta? This feature blocks a lot for us.

@Druotic
Copy link

@Druotic Druotic commented Nov 16, 2016

Any updates? Even a mention that this feature has been abandoned would be better than nothing.

@dsheets
Copy link
Contributor

@dsheets dsheets commented Nov 16, 2016

The feature has not been abandoned. It is still on the roadmap. Thanks for your patience.

@WyseNynja
Copy link

@WyseNynja WyseNynja commented Nov 30, 2016

It is the last day of November. Any update on when this will be in the beta @dsheets?

@dsheets
Copy link
Contributor

@dsheets dsheets commented Dec 1, 2016

Work has begun but is currently delayed behind performance work. Sorry about that. Stay subscribed to get updates. Thanks for your patience!

@gugahoi
Copy link

@gugahoi gugahoi commented Dec 22, 2016

Any updates on this?

@WyseNynja
Copy link

@WyseNynja WyseNynja commented Dec 28, 2016

I've started using https://github.com/avsm/docker-ssh-agent-forward to work around this issue

@WyseNynja
Copy link

@WyseNynja WyseNynja commented Jan 19, 2017

Any updates @dsheets ? Is there an issue tracking the performance work? If that is going to block for much longer, I am planning to spend some time cleaning up https://github.com/avsm/docker-ssh-agent-forward but if docker will have proper support soon I won't bother.

@KFoxder
Copy link

@KFoxder KFoxder commented Jan 19, 2017

+1

@dsheets
Copy link
Contributor

@dsheets dsheets commented Jan 20, 2017

Support for this is planned and has been started but is delayed indefinitely behind other work. @WyseNynja I would recommend cleaning up avsm/docker-ssh-agent-forward in the meantime.

@gerardolima
Copy link

@gerardolima gerardolima commented Feb 26, 2020

The problem still happens today (Docker version 19.03.5, build 633a0ea). Since most of this thread was hijacked by comments related to #410, I'm not sure about the status of this fix, or even if fixing this issue is on the plans for Docker.

Is Support for sharing unix sockets on the development roadmap?

Steps to reproduce the behavior

mount a directory containing unix sockets like so: '-v "/directorywithsockets:/otherdirectory"'
attempt to send data to/from the host/container via the socket

@Alan-R
Copy link

@Alan-R Alan-R commented Feb 27, 2020

@unixneo
Copy link

@unixneo unixneo commented Apr 12, 2020

From here, I cannot get any process on macos outside the docker container to connect to a unix socket inside a Docker container (which is shared outside the container), see for example:

# socat -d -d UNIX-CONNECT:/Users/discourse/discourse/shared/socket-only/run/nginx.http.sock -
2020/04/12 09:46:19 socat[41943] N opening connection to LEN=67 AF=1 "/Users/discourse/discourse/shared/socket-only/run/nginx.http.sock"
2020/04/12 09:46:19 socat[41943] E connect(5, LEN=67 AF=1 "/Users/discourse/discourse/shared/socket-only/run/nginx.http.sock", 67): Connection refused
2020/04/12 09:46:19 socat[41943] N exit(1)

# ls -l /Users/discourse/discourse/shared/socket-only/run/nginx.http.sock
srw-rw-rw-  1 Tim  staff  0 Apr 12 08:42 /Users/discourse/discourse/shared/socket-only/run/nginx.http.sock
@unilynx
Copy link

@unilynx unilynx commented Apr 12, 2020

It is my considered opinion that the only way this ever work completely right is if the Mac moves to a Linux base instead of a BSD base. I'm not holding my breath for that... The system calls to identify the peer on the connection rely on the two endpoints exist in the same process id space. Since Docker processes don't exist as far as MacOS is concerned, that's not happening.

But how is that different from native linux, where the other endpoint may be in a different PID namespace? SCM_CREDENTIALS (pid/uid/gid) is already meaningless. SCM_RIGHTS (file descriptors) might still be useful, although I don't see why you wouldn't be able to use proxy processes on either side of the virtual machine divide if you really wanted this.

Although it's probably still not worth the effort for docker. Most people will only want unix sockets for ssh agent forwarding, and specifically targetting that (properly integrating one of the workarounds listed in the issue) would probably suffice for many.

@Alan-R
Copy link

@Alan-R Alan-R commented Apr 12, 2020

@unilynx Said it just right. Unless MacOS converts to running Linux instead of BSD, his comments are spot on. Pid space, user id space, group id space, etc are all disjoint between MacOS and the containers. Until MacOS converts to Linux this is going to stay the way it is. You can make your own estimation of how probable it is for MacOS to convert to Linux...

But ssh agent forwarding could be made to work with enough chewing-gum, baling wire and duct tape (and maybe a proxy or two).

@Frizlab
Copy link

@Frizlab Frizlab commented Apr 12, 2020

macOS switching to Linux is, I think, not something we can count on, but having a workaround for SSH agent forwarding would be awesome.

@nopdotcom
Copy link

@nopdotcom nopdotcom commented Apr 13, 2020

It would be appreciated if the mechanism for SSH agent forwarding were applicable to other services. Personally, I could really use the GPG agent socket (on macOS, usually at $HOME/.gnupg/S.gpg-agent.extra). Other people might appreciate the X11 socket (on macOS, pointed to by the contents of $DISPLAY; somewhere like /private/tmp/com.apple.launchd.fkXNRcDlhz/org.macosforge.xquartz:0).

@trinitronx
Copy link

@trinitronx trinitronx commented Apr 27, 2020

There now exists an official workaround for this as of Docker for Mac >= 2.2.0.0.

The solution involves using a "magic path" specifically for SSH_AUTH_SOCK: /run/host-services/ssh-auth.sock

Mount this special path into the container via docker run args:

SSH_AUTH_SOCK_MAGIC_PATH='/run/host-services/ssh-auth.sock'
docker run -ti -v ${SSH_AUTH_SOCK_MAGIC_PATH}:${SSH_AUTH_SOCK_MAGIC_PATH} \
           -e SSH_AUTH_SOCK=${SSH_AUTH_SOCK_MAGIC_PATH} \
           some-container/needing-ssh-agent

Note that this does not resolve the main issue here: mounting arbitrary Unix sockets into a container. At least it works for ssh-agent / SSH_AUTH_SOCK!

@xpepermint
Copy link

@xpepermint xpepermint commented May 10, 2020

Hum ... this issue was opened in 2016. Will this be resolved soon?

@Alan-R
Copy link

@Alan-R Alan-R commented May 11, 2020

My advice to anyone new to this issue is to read all the comments. I think they describe it adequately. Specific use cases can be made to work with sufficient effort (as happened recently with ssh forwarding). I would not be shocked to see one or two more specific, narrow use cases taken care of in the next few years. The general problem is unlikely to be fully solved unless MacOS switches to a Linux base. I judge that to be highly unlikely unless something drastic happens in or to Apple. If you just want it to behave like a FIFO, then that's not so hard. However, UNIX domain sockets have some specific and useful properties which would be troublesome to implement between two dissimilar operating systems.

@adamnovak
Copy link

@adamnovak adamnovak commented May 12, 2020

@fred-vogt
Copy link

@fred-vogt fred-vogt commented May 19, 2020

@adamnovak - There is still no FIFO support as far as I know.

You can generally use a pair of FIFOs and a Mac listening port to proxy traffic between a pair of domain sockets.

This is facilitated by a pair of netcats in each environment.

This is as convoluted as it sounds - but it does work reliably.

Host:

# Mac - docker run wrapper - before launching the container
_PROXY_FIFO=`mktemp -u`
mkfifo $_PROXY_FIFO

PROXY_PORT=$(python -c 'import socket; s = socket.socket(); s.bind(("", 0)); print(s.getsockname()[1]); s.close()')

_PROXIED_SOCKET=
nc -lk 127.0.0.1 "$PROXY_PORT" <"$_PROXY_FIFO" | nc -U "$_PROXIED_SOCKET" >"$_PROXY_FIFO" &

trap "exit" INT TERM ERR
trap "{ kill 0; rm -f $_PROXY_FIFO; }" EXIT

Container: -e PROXY_PORT=...

# Called from inside the container entrypoint
_PROXY_FIFO=`mktemp -u`
mkfifo "$_PROXY_FIFO"

_PROXIED_SOCKET=
nc host.docker.internal "$PROXY_PORT" <"$_PROXY_FIFO" | nc -Ulk "$_PROXIED_SOCKET" >"$_PROXY_FIFO" &

Exhibit A Keybase client

NOTE: the client & server keybase versions must be an exact version match

keybase client usage from inside the container:

keybase --no-auto-fork --local-rpc-debug-unsafe A decrypt --infile ...

Unix domain sockets:

# Mac unix socket:
_PROXIED_SOCKET="${HOME}/Library/Group Containers/keybase/Library/Caches/Keybase/keybased.sock"

# Container unix socket:
_PROXIED_SOCKET="${HOME}/.config/keybase/keybased.sock"

Routing:

keybase client in container:
  "sends" - container:/keybased.sock => container:fifo => host:ephemeral-tcp-port
  "recv"  - host:ephemeral-tcp-port => ~/.config/keybase/keybased.sock
keybase server on Mac:
  "recv"  - host:ephemeral-tcp-port => host:/keybased.sock => host:fifo
  "sends" - host:fifo => host:ephemeral-tcp-port
@dancojocaru2000
Copy link

@dancojocaru2000 dancojocaru2000 commented Jun 7, 2020

@unilynx

Most people will only want unix sockets for ssh agent forwarding

Most, but certainly not all.

The main issue for my personal use case is that I don't want ports exposed on the host. Instead, I would like a container port to be exposed on a UNIX socket and then use nginx to forward that UNIX socket to a certain hostname:

  • app1.serveraddress.com
  • app2.serveraddress.com
  • app3.serveraddress.com

All these domains would point to the same computer and nginx would forward to a different UNIX socket based on the different hostname.

Exposing the 3 apps on ports - let's say - 7001, 7002 and 7003 and then doing some forwarding would mean that one would be able to access app1 from app2.serveraddress.com:7001, which isn't desirable.

@benlangfeld
Copy link

@benlangfeld benlangfeld commented Jun 7, 2020

@unilynx

Most people will only want unix sockets for ssh agent forwarding

Most, but certainly not all.

The main issue for my personal use case is that I don't want ports exposed on the host. Instead, I would like a container port to be exposed on a UNIX socket and then use nginx to forward that UNIX socket to a certain hostname:

  • app1.serveraddress.com

  • app2.serveraddress.com

  • app3.serveraddress.com

All these domains would point to the same computer and nginx would forward to a different UNIX socket based on the different hostname.

Exposing the 3 apps on ports - let's say - 7001, 7002 and 7003 and then doing some forwarding would mean that one would be able to access app1 from app2.serveraddress.com:7001, which isn't desirable.

You can use an overlay network for that, like Calico.

@dancojocaru2000
Copy link

@dancojocaru2000 dancojocaru2000 commented Jun 7, 2020

How exactly would an overlay network help in my case?

@unilynx
Copy link

@unilynx unilynx commented Jun 7, 2020

@unilynx

Most people will only want unix sockets for ssh agent forwarding

Most, but certainly not all.

The main issue for my personal use case is that I don't want ports exposed on the host. Instead, I would like a container port to be exposed on a UNIX socket and then use nginx to forward that UNIX socket to a certain hostname:

Why not publish the ports bound to only 127.0.0.1 ?

(I haven't personally tested with 127.0.0.1, but I do run a few dockers with their ports published to an alias 10.x.x.x address I've assigned to the localhost interface)

@dancojocaru2000
Copy link

@dancojocaru2000 dancojocaru2000 commented Jun 7, 2020

The port would still be bounded.

Still, haven't thought about that approach! Thanks for reminding me I can probably do that!

@benlangfeld
Copy link

@benlangfeld benlangfeld commented Jun 7, 2020

How exactly would an overlay network help in my case?

Your nginx instance would have interfaces on both the overlay and the external networks.

@trinitronx
Copy link

@trinitronx trinitronx commented Jun 22, 2020

I definitely see another use case with $HOME/.gnupg/S.gpg-agent* sockets:

Building software inside a Docker container, and GPG signing the resulting build artifacts with a Yubikey plugged into the host OS. For example: Building a Debian package withdebuild + export DEBUILD_DPKG_BUILDPACKAGE_OPTS="-sa -k${DEBSIGN_KEYID}"

This would need access to the gpg-agent socket for signing the package & .dsc, .changes, and .buildinfo files.

@rkettelerij
Copy link

@rkettelerij rkettelerij commented Dec 9, 2020

This is indeed very much needed in order to securely use GPG within Docker. GnuPG gpg-agent offers an extra domain socket which you can forward from the host to your container and hence use GPG inside the container. Lot's of stuff (e.g. pass or gopass, password manager) integrates with GPG so this isn't an exotic use case IMHO.

Since SSH forwarding works now, what would it take to make this work for GPG?

@AndreaCensi
Copy link

@AndreaCensi AndreaCensi commented Dec 20, 2020

I am not sure I understand one thing: /var/run/docker.sock is a socket, right? and that one socket can be mounted in the containers, arbitrarily deep. Is that a special case that is different from user-created sockets?

Mahoney added a commit to Mahoney/docker-lifecycle-listener that referenced this issue Jan 10, 2021
In principle a unix socket would be a more secure way of communicating
as there would be no exposed port that could be hit by other machines on
the same network.

However, docker/for-mac#483 means it won't
work on docker desktop for Mac, and as one of the primary motivators for
this project is working around
docker/for-mac#155 by enabling
https://github.com/Mahoney-forks/docker-tuntap-osx that's a blocker.
@stephen-turner
Copy link
Contributor

@stephen-turner stephen-turner commented Jan 12, 2021

Sorry to disappoint everyone, but I'm closing this ticket as "Won't Fix". The main use case has been solved in #410. As described in recent comments, the general case is too hard / impossible, and is realistically never going to be done.

@Vanuan
Copy link

@Vanuan Vanuan commented Jan 13, 2021

@stephen-turner what's the current state of osxfs? Is it stiil being used? There's been a talk about open sourcing it several years ago.

@stephen-turner
Copy link
Contributor

@stephen-turner stephen-turner commented Jan 14, 2021

It's still in the product at the moment, but is being replaced because of a lot of problems with it, particularly high CPU usage. There is no intent to open source it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet