New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Document how to connect to Docker host from container #1143

Closed
bkad opened this Issue Jul 5, 2013 · 262 comments

Comments

Projects
None yet
@bkad

bkad commented Jul 5, 2013

I had some trouble figuring out how to connect the docker host from the container. Couldn't find documentation, but did find irc logs saying something about using 172.16.42.1, which works.

It'd be nice if this behavior and how it's related to docker0 was documented.

@mhennings

This comment has been minimized.

Contributor

mhennings commented Jul 7, 2013

when you look inside network.go you find that docker probes for internal networks that are not routed.

first 172.16.42.1 is guessed as a bridge address then others.

So documenting this wont help much. Its a dynamic scheme you can not rely on.

I think what you require is more a way to define the addresses used for bridge and client.
could that be?

@cespare

This comment has been minimized.

Contributor

cespare commented Jul 8, 2013

I think the requirement is clear from the issue title. There needs to be an easy and well-documented way to talk to the host from the container, however it's implemented.

@ghost ghost assigned metalivedev Jul 10, 2013

@ebensing

This comment has been minimized.

ebensing commented Aug 8, 2013

+1 It would be really nice to have a good way to connect to the host system

@shykes

This comment has been minimized.

Collaborator

shykes commented Aug 8, 2013

+1, the 1.0 branch will define an introspection API so that each container can interact with the host in a scoped and controlled way.

This is currently planned for 0.8. 


@solomonstre
@getdocker

On Thu, Aug 8, 2013 at 1:10 PM, E.J. Bensing notifications@github.com
wrote:

+1 It would be really nice to have a good way to connect to the host system

Reply to this email directly or view it on GitHub:
#1143 (comment)

@gerhard

This comment has been minimized.

gerhard commented Sep 18, 2013

So how can I connect to the docker host from within a container? I am trying to connect to a docker container via the host port rather than the container's private IP.

@jpetazzo

This comment has been minimized.

Contributor

jpetazzo commented Sep 23, 2013

@gerhard: an introspection API is planned for 0.8. Meanwhile, if you want to access the Docker API from the containers, you can setup Docker to listen on the IP address of the Docker bridge.

To do that, you would:

  • create a bridge

    ip link add docker0 type bridge

  • assign an IP address to it

    ip link set docker0 up
    ip addr add 172.17.0.1/16 dev docker0

  • start docker and bind the API to the bridge

    docker -d -H 172.17.0.1:4242

Now you can access the Docker API from your containers.

@shykes

This comment has been minimized.

Collaborator

shykes commented Dec 2, 2013

Currently (version 0.7) docker does not reliably support granting unlimited access to its own control socket to one of its containers. The workarounds explained in this thread are hacks which are not guaranteed to work, and if they do might break at any time - please don't use them in production or expect us to support them. Since there is no official feature to document, this doc issue can't be fixed.

To discuss hacks and workarounds for missing features, I recommend either the docker-user mailing list, or the #docker irc channel on Freenode.

Happy hacking

@shykes shykes closed this Dec 2, 2013

@cespare

This comment has been minimized.

Contributor

cespare commented Dec 2, 2013

@shykes Is there another issue that tracks the creation of such a feature, in that case?

@cespare

This comment has been minimized.

Contributor

cespare commented Dec 2, 2013

By the way, to give motivation for such a feature: this is useful when locally testing a server (where I would've used vagrant in the past) and I want to connect a server in the container to a database or other server running on my dev machine (the docker host).

@solomonstre

This comment has been minimized.

solomonstre commented Dec 2, 2013

I am already sold on the value of this feature :)

On Mon, Dec 2, 2013 at 9:09 AM, Caleb Spare notifications@github.com
wrote:

By the way, to give motivation for such a feature: this is useful when locally testing a server (where I would've used vagrant in the past) and I want to connect a server in the container to a database or other server running on my dev machine (the docker host).

Reply to this email directly or view it on GitHub:
#1143 (comment)

@misho-kr

This comment has been minimized.

misho-kr commented Jan 18, 2014

I am on Fedora 20 with docker 0.7.2, setting up Docker UI. I had to open the port on which docker daemon listens so the firewall does not block it:

  • firewall-cmd --permanent --zone=trusted --add-interface=docker0
  • firewall-cmd --permanent --zone=trusted --add-port=4243/tcp

After that docker-ui was able to connect to the docker daemon control socket.

HTH
This is a clear and legitimate need for such feature.

@jonasfj

This comment has been minimized.

jonasfj commented Mar 19, 2014

I'm sorry, if I'm keeping a die hard thread alive.

The title of this issues says: "How to connect to host from docker container".
I don't see how this relates to the docker inspect feature. The inspect feature is used on the host-side to find the IP of the container, if I'm not mistaken.

I think the issue by bkad is finding the host IP from within the container. Granted I'm not networking wizard, but isn't is fairly safe to assume that the ip gateway (from inside the container) maps to the host.
(Assuming one doesn't configure a bridge setup or something).

Using the gateway for 0.0.0.0 from netstat -nr I certainly had no problems reaching a server running on my host machine. I suspect that the gateway ip is static (once docker is started), can anybody confirm that?

A alternative would be to pass my hosts public ip to the container using environment variables, but the public IP may not be static. And whilst hostname might work better in production, they are hard to use locally.

I still would prefer a way to call from docker container to host through the loopback and appear as 127.0.0.1 on the host. Or if security is a concern another loopback device that always has the same ip.
Or maybe the thing I found doesn't expose my communication to the public? Like said I'm no network wizard :)
Note, if using ip-gateway to call docker host is the "right way", can't we document it?

For those looking to find the gateway ip from container /proc/net/route is probably the right place to read it.

The Motivation, for this feature includes various meta-data services. Exposing things from ec2 meta-data service would be nice. Distribution of credentials and more complex structured data that doesn't fit into environment variables.

@jpetazzo

This comment has been minimized.

Contributor

jpetazzo commented Mar 24, 2014

@jonasfj: there is actually an easier way, now that Docker supports bind-mounting files from host to container. You can bind-mount the Docker control socket, i.e. : docker run -v /var/run/docker.sock:/var/run/docker.sock …; this is easier than fiddling with networking rules.

@kontrafiktion

This comment has been minimized.

kontrafiktion commented Mar 26, 2014

I am not sure how the docker socket helps. I still think the issue should be reopened, there is no documentation for the following scenario:

  1. On the 'host' a service runs on port 8080 (say 'etcd')
  2. From that host a docker container is started
  3. How can the service on port 8080 on the host be reached from the docker container? What would be the URL/IP?

@jpetazzo How does setting the docker.sock come in to play to solve the above problem?

@jonasfj

This comment has been minimized.

jonasfj commented Mar 26, 2014

@vrvolle, exposing docker.sock doesn't solve the original issue described by @bkad.
But one would expose a different unix domain socket for communication between host and docker-container.

For example, if you wanted to expose mysql from the host you would expose the mysql socket: /tmp/mysql.sock.

Or if you like me have a metadata API through which containers should be able to query the host for various useful things, you create your own unix domain socket and expose it to the container. HTTP over unix domain sockets should work very well. Also you don't have all the network configuration issues and security issues.

@kontrafiktion

This comment has been minimized.

kontrafiktion commented Mar 26, 2014

just had to read up upon 'unix domain sockets'.
but:

http://stackoverflow.com/questions/14771172/http-over-af-unix-http-connection-to-unix-socket

claims there is no URL and therefore no usual client can use that mechanism out of the box.

On the other hand:

http://stackoverflow.com/questions/14771172/http-over-af-unix-http-connection-to-unix-socket

shows that it is somehow possible to use such a socket.

But still I would like to simple be able to have an IP adress and port, which a program inside a docker could simply use. I will -- for now -- use

 netstat -nr | grep '^0\.0\.0\.0' | awk '{print $2}'

Should I create a new issue or can someone reopen this one.

@jonasfj

This comment has been minimized.

jonasfj commented Mar 26, 2014

@vrvolle
I'm not networking guy, but I imagine that there is some tricks one can do to proxy localhost into the container over a unix socket and then inside the container proxy unix socket to the containers-localhost (loopback device)...

It would be nice to document how to do that. But it seems that it's not necessarily a feature docker needs to active support.

@jpetazzo

This comment has been minimized.

Contributor

jpetazzo commented Apr 2, 2014

There are multiple ways to address that, depending on what exactly you want to achieve.

If you want to connect to a service running on the host (any kind of service, e.g. some API or database that would be running straight on the host), you have to figure out the IP address of the host.

One way is to rely on the fact that the Docker host is reachable through the address of the Docker bridge, which happens to be the default gateway for the container. In other words, a clever parsing of ip route ls | grep ^default might be all you need in that case. Of course, it relies on an implementation detail (the default gateway happens to be an IP address of the Docker host) which might change in the future.

Another way is to use the Docker API to retrieve that information. Then, the problem becomes "how do I connect to the Docker API from a container?", and a potential solution (used by many, many containers out there!) is to bind-mount /var/run/docker.sock from the host to the container. This has a big downside: the API becomes available in full to the container, which might do bad things with it.

In the long term, Docker will expose a better introspection API, allowing to access that information without giving away too much privilege to the containers.

TL,DR: short term, check the default route of the container. Long term, there will be an awesome introspection API.

@harmnprt

This comment has been minimized.

harmnprt commented May 14, 2014

i have also problem due to upstart. i can't find anything in /var/run/docker.sock. and used command "-v /etc/run/docker.sock:/etc/run/docker.sock" but nothing happened.
i think is issue due to some new updates about capabilities of kernel. please give me brief note on this issue. and full command to solve this. thanks

@michaelneale

This comment has been minimized.

Contributor

michaelneale commented May 16, 2014

use -v /var/run/docker.sock - not /etc (which is normally reserved for conf files).

@c4milo

This comment has been minimized.

c4milo commented Jun 12, 2014

any update about this in the new 1.0.0 release?

@michaelneale

This comment has been minimized.

Contributor

michaelneale commented Jun 13, 2014

There is nsenter - but I think the encouraged way is to run sshd at this
stage.

On Friday, June 13, 2014, Camilo Aguilar notifications@github.com wrote:

any news about this in the new 1.0.0 release?


Reply to this email directly or view it on GitHub
#1143 (comment).

Michael D Neale
home: www.michaelneale.net
blog: michaelneale.blogspot.com

@Sepero

This comment has been minimized.

Sepero commented Jun 15, 2014

vrvolle, thanks for that. A lot of people like us are looking for a little tidbit like this

netstat -nr | grep '^0\.0\.0\.0' | awk '{print $2}'
@elgalu

This comment has been minimized.

Contributor

elgalu commented Jul 8, 2014

Docker auto updating /etc/hosts on every container with the host IP, e.g. 172.17.42.1 and calling it for example dockerhost would be a convenient fix.
I guess for now we are stuck with netstat -nr | grep '^0\.0\.0\.0' | awk '{print $2}'

@aceofspades

This comment has been minimized.

aceofspades commented Jul 16, 2014

+1 for dockerhost in /etc/hosts

@Dieterbe

This comment has been minimized.

Dieterbe commented Jul 16, 2014

+1 for dockerhost in /etc/hosts, sounds like a good idea

@Sepero

This comment has been minimized.

Sepero commented Jul 17, 2014

Editing a file in an image should never be done UNLESS specifically give an argument or flag to do so. Also, it isn't mandatory that 100% of images will follow the LSB, so there might not even be an /etc directory. The filename to use for containing host IP should also be specified by command argument.

docker --host /etc/hosts
@deltabweb

This comment has been minimized.

deltabweb commented Jun 21, 2017

@halfnibble
I haven't looked at the documentation of networks in details but my understanding is that a new network on your host machine is created where :

  • IP of the "dockerhost" is 192.168.0.1
  • IP of the first container is 192.168.0.2
  • IP of the second container is 192.168.0.3
  • and so on ...

Starting from there, everything should work as if you had physical machines connected together on a local network :

  • machines can connect to each other
  • and you should even be able to use ping 192.168.0.2 from the host - this will only work if your container responds to pings though.

So to answer your question, I think that your apps on the host machine will need to respond to 192.168.0.X (depending on the container trying to connect).

@nooperpudd

This comment has been minimized.

nooperpudd commented Jun 22, 2017

@deltabweb how cloud I access the server host port? >>> Connection refused

@deltabweb

This comment has been minimized.

deltabweb commented Jun 22, 2017

@nooperpudd Just to be sure : you are trying to access an application running on the host from a container right ?
I would first check if the application allows incoming connections from outside (0.0.0.0 and not just localhost). And maybe also make sure that you don't have a firewall blocking the connection ?

@HosseinAgha

This comment has been minimized.

HosseinAgha commented Jun 22, 2017

@nooperpudd if you are using Docker for mac you cannot use host mode, @deltabweb solution is also not working for me (my servers are all listening to 0.0.0.0 and my host machine firewalls were turned off but I get Connection refused every time). After about 2 days of try and error the only way that I found to fix this issue is the bellow script:

#!/bin/bash
export DOCKERHOST=$(ifconfig | grep -E "([0-9]{1,3}\.){3}[0-9]{1,3}" | grep -v 127.0.0.1 | awk '{ print $2 }' | cut -f2 -d: | head -n1)

# You should use DOCKERHOST env variable in your `docker-compose.yml` 
# and put it everywhere you want to connect to `localhost` of the host machine
docker-compose $@

Only problem with this approach is if your IP changes after running your containers you should run them again and they cannot find your new IP.

@eddytruyen

This comment has been minimized.

eddytruyen commented Sep 23, 2017

A remarkable observation for me is that the command, which is executed when starting docker containers, cannot connect to the container via its host IP address. However when that command does not perform such connection attempts and finishes executing successfully , the container is then able to reach itself via its host IP address.

I made this observation when trying to expose the endpoints of NoSql database cluster instances to clients outside the swarm cluster. After all, these endpoints need to be configured with private or public IP addresses of the VM in order for the external client to reach them. Cassandra is however designed in such a way that when it starts it tries to immediately connect with the host IP address (set as CASSANDRA_BROADCAST_ADDRESS environment variable -- see below) and therefore fails. On the other hand, Mongodb replicasets nodes are first all started in a clean state, and then a separate initating command is executed such that the primary and secondary nodes can form a replicaset.

Below you see a detailed account of this observation for cassandra (I create these with docker swarm but the same problem appears in docker run -d (in NAT mode, thus without --net=host option)

  1. On the one hand, a container created by
docker service create  --name cassandra-service
--publish mode=host,target=7000,published=7000,protocol=tcp 
-e CASSANDRA_SEEDS=host IP address  -e CASSANDRA_BROADCAST_ADDRESS=host IP address

fails with the message that it cannot connect to the listen address: <host IP address>:7000

  1. On the other hand, a container attached to an overlay network, created by
docker service create  --network cassandra-net --name cassandra-service
-e CASSANDRA_SEEDS=cassandra-service  -e CASSANDRA_BROADCAST_ADDRESS=cassandra-service

starts correctly and at the same time I can connect to the host ip address on any port that is exposed in the Dockerfile of the cassandra:2.0 image:

$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED              STATUS              PORTS                                         NAMES
07603a75a379        cassandra:2.0       "/docker-entrypoin..."   About a minute ago   Up About a minute   7000-7001/tcp, 7199/tcp, 9042/tcp, 9160/tcp   cassandra-service-1.1.m243u97zku15w08m6puytdngs

$ docker exec -it 1e61ec16f8d0 bash
root@1e61ec16f8d0:/# cqlsh 172.17.13.151
Connected to Test Cluster at 172.17.13.151:9160.
[cqlsh 4.1.1 | Cassandra 2.0.17 | CQL spec 3.1.1 | Thrift protocol 19.39.0]

Similarly, the same can be observed during the creation of a second cassandra node

  1. If I create a second cassandra container on another node by
docker service create  --network cassandra-net --name cassandra-service-2
-e CASSANDRA_SEEDS=172.17.13.151  -e CASSANDRA_BROADCAST_ADDRESS=cassandra-service-2

the container fails with the runtime exception that it cannot gossip with the seed:

java.lang.RuntimeException: Unable to gossip with any seeds
        at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1322)
        at org.apache.cassandra.service.StorageService.checkForEndpointCollision(StorageService.java:457)
  1. On the other hand, if I create a cassandra container via docker run -d, I can reach the seed node via its host IP address:
$ docker run -d cassandra:2.0
d87a79cc3de8cd7e4cf40284d1eca91ceb660581cc71082fe64a6b84a09fbd77
$ docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                                         NAMES
d87a79cc3de8        cassandra:2.0       "/docker-entrypoin..."   3 seconds ago       Up 2 seconds        7000-7001/tcp, 7199/tcp, 9042/tcp, 9160/tcp   trusting_ardinghelli
$ docker exec -it d87a79cc3de8 bash
root@d87a79cc3de8:/# cqlsh 172.17.13.151
Connected to Test Cluster at 172.17.13.151:9160.
[cqlsh 4.1.1 | Cassandra 2.0.17 | CQL spec 3.1.1 | Thrift protocol 19.39.0]
Use HELP for help.
cqlsh>

Specifically for Cassandra, you solve this problem by turning auto bootstrapping of cassandra nodes off. You do that by setting auto_bootstrap to false in /etc/cassandra/cassandra.yaml using an entrypoint command in Compose V3:

version: '3'
services:
  cassandra-1:
    image: cassandra:2.0
    entrypoint:
    - "sh"
    - "-c"
    - "echo auto_bootstrap: false >> /etc/cassandra/cassandra.yaml; /docker-entrypoint.sh cassandra -f"
    environment:
      CASSANDRA_BROADCAST_ADDRESS: 172.17.13.151
    volumes:
    - volume1:/var/lib/cassandra
    ports:
    - "7000:7000"
    - "7001"
    - "7199"
    - "9042:9042"
    - "9160:9160"

and then manually start cassandra nodes by executing docker exec -it <container id> nodetool rebuild.

@RDeluxe

This comment has been minimized.

RDeluxe commented Oct 23, 2017

I could use this feature in development, well ...

@acuthbert

This comment has been minimized.

acuthbert commented Nov 23, 2017

@jpetazzo We develop PHP solutions in teams on a mix of platforms. Our debugger (xdebug) needs to connect back to the IDE on the host. On Windows and Linux this works 'out of the box' but on mac our developers have to change the xdebug.ini file to specifically mention their local IP. But the Dockerfile is under source control... queue constant conflicts and swearing as developers clash over editing this file. Yes there are scriptable workarounds but why does docker for windows and mac have docker.for.win.localhost and docker.for.mac.localhost? It's partially helpful but we still need scripts to detect which platform we're on to set this up right. It just seems so much more complicated that it should be. Please reconsider this feature. Docker can be a steep learning curve but issues like this leave your users searching in disbelief on google for hours on end.

@rskuipers

This comment has been minimized.

rskuipers commented Jan 2, 2018

Checking the https://docs.docker.com/docker-for-mac/networking/#use-cases-and-workarounds page helped, using docker.for.mac.localhost worked for us :)

@moby moby deleted a comment from rskuipers Jan 2, 2018

@jedwards1211

This comment has been minimized.

jedwards1211 commented Jan 16, 2018

For better or worse, docker-compose is still by far the easiest way I know of to spin up a Selenium Hub and headless Firefox/Chrome grid nodes for testing a web server running on a local dev machine or in CI (launching the web server in a docker container is just too slow to be convenient for development). It's not at all what Docker was intended for, but Docker's the best tool for that job, and it's Docker's own fault 😆 Except the only problem is easily figuring out the host ip in a way that works on any OS.

@jzavrl

This comment has been minimized.

jzavrl commented Jan 24, 2018

@rskuipers can you explain what exactly did you do with docker.for.mac.localhost? I'm trying to make requests from inside of my containers to resolve to the host machine. My containers are running on traefik which means I can access them through domain.docker.localhost, but if I try to access an URL that starts with that from inside my container it's not resolving.

Currently what I did, is I added this to my docker-compose.yml, which adds a line to /etc/hosts so that the domain resolves nicely:

extra_hosts: - "domain.docker.localhost:172.18.0.1"

The IP is the host IP from within my container, which I can get by using ip route | awk '/^default via /{print $3}'. But I wouldn't like to hardcode that if possible ...

@rskuipers

This comment has been minimized.

rskuipers commented Jan 25, 2018

@jzavrl All I did was use docker.for.mac.localhost to have my HTTP requests go through a proxy running on the host. I don't use any other layers aside from docker-compose.

@jzavrl

This comment has been minimized.

jzavrl commented Jan 25, 2018

That's exactly what I'm interested in. What sort of changes specifically you had to make?

@rskuipers

This comment has been minimized.

rskuipers commented Jan 25, 2018

@jzavrl None :P it just worked.

@jzavrl

This comment has been minimized.

jzavrl commented Jan 25, 2018

I don't get it, what did you do with docker.for.mac.localhost then?

@rskuipers

This comment has been minimized.

rskuipers commented Jan 25, 2018

@jzavrl I used that instead of the IP to connect to. So docker.for.mac.localhost:8888

@jzavrl

This comment has been minimized.

jzavrl commented Jan 25, 2018

Ahhhhhh, now that's starting to make sense now. Will try this then. Cheers @rskuipers.

@wuzhefang

This comment has been minimized.

wuzhefang commented Mar 11, 2018

just use the "en0" 's ip on your computer.

for example

ssh thefun@192.168.1.100

'192.168.1.100' maybe from your router's DHCP service.

@NdubisiOnuora

This comment has been minimized.

NdubisiOnuora commented Mar 15, 2018

@acuthbert, thanks for your the suggestion.
docker.for.win.localhost works for me in Docker for Windows. There is hope for Docker and Windows yet. 😣

@christhomas

This comment has been minimized.

christhomas commented Mar 16, 2018

there is little technical reason why this cannot be done and satisfy the 90% of the people on this thread, the corner cases and situations where it doesn't really work, the people who are developing in that situaton could be satisfied with a simple set of "use-cases" which explain what scenarios its likely to not work.

This is mostly just political trash and not actual technical reasoning here. I'm hoping one of the other container engines picks up and I can swap kubernetes to using that instead. Then I won't have to deal with this rubbish anymore.

@kepo4ka

This comment has been minimized.

kepo4ka commented Mar 24, 2018

@NdubisiOnuora, what type of your application? web-application?

I have 2 console apps (tcp-server in host and tcp-client in container).
Because they use tcp, i need exactly IP (docker.for.win.localhost do no fit, because it's domain).

For example, Which ip:port i must set in tcp-client, If i set ip:port 127.0.0.1:9595 in tcp-server?

@orf

This comment has been minimized.

orf commented Mar 24, 2018

Just resolve the domain to an IP address?

@kepo4ka

This comment has been minimized.

kepo4ka commented Mar 24, 2018

@orf,
I want use this code in C#:
IPAddress hostAddr = Dns.Resolve("docker.for.win.localhost").AddressList[0];
But before that I try ping to docker.for.win.localhost, but don't see that, error: Ping request could not find host docker.for.win.localhost. Please check the name and try again.
My Dockerfile:
FROM microsoft/windowsservercore
ADD . /
ENTRYPOINT powershell ping docker.for.win.localhost

@colinmollenhour

This comment has been minimized.

colinmollenhour commented Aug 28, 2018

In case anyone missed it I believe the solution as of 18.03 is host.docker.internal although for some reason this only works on Docker for Windows!? Why not others?

EDIT: Didn't see that comments were collapsed by Github... 🤦‍♂️

@lukasmrtvy

This comment has been minimized.

lukasmrtvy commented Dec 4, 2018

Works for me:
docker run --rm -it --add-host "docker.for.localhost:$(ip -4 addr show docker0 | grep -Po 'inet \K[\d.]+')" alpine:latest ping docker.for.localhost

@colinmollenhour

This comment has been minimized.

colinmollenhour commented Dec 4, 2018

@lukasmrtvy That works for a shell, but how about for docker-compose.yml?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment