New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Possible to use ip addresses and DNS names for docker containers in OS X bridge network #3402

Open
rmelick opened this Issue May 4, 2016 · 5 comments

Comments

Projects
None yet
3 participants
@rmelick

rmelick commented May 4, 2016

Hi All,

Summary

I would like to be able to connect to my docker containers by their host name from my host OS X laptop. I do not know if this is supported, or what network configuration options would need to be enabled.

Background

I installed docker using the Docker Toolbox (https://docs.docker.com/mac/step_one/)

Versions

  • docker-machine: docker-machine version 0.7.0, build a650a40
  • docker: Docker version 1.11.0, build 4dc5990

Hosts

  • OS X laptop: rmelick-macbook
  • virtualbox docker machine: default
  • docker containers: amb-server, amb-agent-1

Compose File

I am trying to deploy the following compose file

version: '2'

services:
  amb-server:
    hostname: amb-server
    container_name: amb-server
    networks:
    - cloud
    image: centos:7
    command: [/bin/sh, -c, while true; do echo hello world; sleep 1000; done]
    ports:
    - 8080:8080
    depends_on:
    - amb-agent-1
    environment:
      TRACE: 1
  amb-agent-1:
    hostname: amb-agent-1
    container_name: amb-agent-1
    networks:
    - cloud
    image: centos:7
    command: [/bin/sh, -c, while true; do echo hello world; sleep 1000; done]
    environment:
      TRACE: 1

networks:
  cloud:
    driver: bridge

Problem

After deploying that compose file, I would like to be able to access the two docker images by the hostnames (amb-server.docker_cloud and amb-agent-1.docker_cloud) from my host laptop rmelick-macbook. When I try to ping, I see this error:

$ ping amb-agent-1.docker_cloud
ping: cannot resolve amb-agent-1.docker_cloud: Unknown host

If I log into one of the docker containers, everything works fine

docker exec -it amb-server bash

[root@amb-server /]# ping amb-agent-1
PING amb-agent-1 (172.18.0.2) 56(84) bytes of data.
64 bytes from amb-agent-1.docker_cloud (172.18.0.2): icmp_seq=1 ttl=64 time=0.071 ms
64 bytes from amb-agent-1.docker_cloud (172.18.0.2): icmp_seq=2 ttl=64 time=0.082 ms

If I use this ip address from my host laptop, it also fails

$ ping 172.18.0.2
PING 172.18.0.2 (172.18.0.2): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2

If I log on to the docker-machine (default), the dns lookup fails, but the ping by ip address functions correctly

docker-machine ssh default

docker@default:~$ ping 172.18.0.2
PING 172.18.0.2 (172.18.0.2): 56 data bytes
64 bytes from 172.18.0.2: seq=0 ttl=64 time=0.073 ms
64 bytes from 172.18.0.2: seq=1 ttl=64 time=0.093 ms

docker@default:~$ ping amb-agent-1.docker_cloud
ping: bad address 'amb-agent-1.docker_cloud'

Is there any way to expose the dns and ip addresses of this bridge network to my host laptop? The documentation (https://docs.docker.com/engine/userguide/networking/dockernetworks/#a-bridge-network) seems to say that I can only expose specific ports when using a bridge network.

The real reason I want to do this is we're attempting to deploy hadoop within docker, and the internal hadoop services are all configured with the hostnames of the different docker containers. I would like to be able to run code outside of a docker container that can connect with Hadoop running inside docker. If this doesn't work, it seems I will have to run all of my code inside docker containers that join that network.

@skitching-um

This comment has been minimized.

Show comment
Hide comment
@skitching-um

skitching-um Dec 6, 2017

Does anyone have a solution for this problem? I am trying to doing something similar: running a Kafka broker instance (one-node cluster) in a container and connecting to it from the host (mac-os). The kafka client lib connects to the broker itself by localhost:exported-port fine, but then queries it for the "list of node addresses", which are of form "containername:port" which it then cannot resolve in the host environment.

I wish to be able to execute automated integration tests against this kafka broker; I have tests that work fine for other software in docker containers, but cannot get Kafka tests working because of this "get list of node addresses" operation which returns addresses that the caller cannot resolve.

skitching-um commented Dec 6, 2017

Does anyone have a solution for this problem? I am trying to doing something similar: running a Kafka broker instance (one-node cluster) in a container and connecting to it from the host (mac-os). The kafka client lib connects to the broker itself by localhost:exported-port fine, but then queries it for the "list of node addresses", which are of form "containername:port" which it then cannot resolve in the host environment.

I wish to be able to execute automated integration tests against this kafka broker; I have tests that work fine for other software in docker containers, but cannot get Kafka tests working because of this "get list of node addresses" operation which returns addresses that the caller cannot resolve.

@tgudlek

This comment has been minimized.

Show comment
Hide comment
@tgudlek

tgudlek Mar 19, 2018

@skitching-um came here with the exact problem you were having. How did you end up solving it?

tgudlek commented Mar 19, 2018

@skitching-um came here with the exact problem you were having. How did you end up solving it?

@skitching-um

This comment has been minimized.

Show comment
Hide comment
@skitching-um

skitching-um Mar 20, 2018

My solution is documented here: http://moi.vonos.net/java/network-name-mapper/

In summary, I use Java reflection to override java.net.INetAddress.getAddress() and map the docker container name to the appropriate IP address. It is somewhat of a hack, but does the job..

skitching-um commented Mar 20, 2018

My solution is documented here: http://moi.vonos.net/java/network-name-mapper/

In summary, I use Java reflection to override java.net.INetAddress.getAddress() and map the docker container name to the appropriate IP address. It is somewhat of a hack, but does the job..

@tgudlek

This comment has been minimized.

Show comment
Hide comment
@tgudlek

tgudlek Mar 20, 2018

Thanks for the follow-up!

I've ended up moving away from homebrew version of Docker and installed Docker for Mac, set ADVERTISED_HOST=localhost, and used localhost:9092 as a broker hostname which seems to be working as well.

tgudlek commented Mar 20, 2018

Thanks for the follow-up!

I've ended up moving away from homebrew version of Docker and installed Docker for Mac, set ADVERTISED_HOST=localhost, and used localhost:9092 as a broker hostname which seems to be working as well.

@skitching-um

This comment has been minimized.

Show comment
Hide comment
@skitching-um

skitching-um Mar 20, 2018

@tgudlek Using localhost works when the cluster is a single node and the only thing that talks to the Kafka broker is the app outside the container environment. If you have another container which also talks to Kafka (eg a kafka-connector instance, or a schema-registry) then using "localhost" as the advertised address for the broker node no longer works for obvious reasons.

skitching-um commented Mar 20, 2018

@tgudlek Using localhost works when the cluster is a single node and the only thing that talks to the Kafka broker is the app outside the container environment. If you have another container which also talks to Kafka (eg a kafka-connector instance, or a schema-registry) then using "localhost" as the advertised address for the broker node no longer works for obvious reasons.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment