New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

network not reachable on multi-host overlay network #644

Closed
baopx opened this Issue Oct 13, 2015 · 11 comments

Comments

Projects
None yet
4 participants
@baopx

baopx commented Oct 13, 2015

Environment:

docker version: experimental:

-bash-4.2# docker version
Client:
Version: 1.9.0-dev
API version: 1.21
Go version: go1.4.2
Git commit: 8c5d511
Built: Wed Sep 2 16:25:21 UTC 2015
OS/Arch: linux/amd64
Experimental: true

Server:
Version: 1.9.0-dev
API version: 1.21
Go version: go1.4.2
Git commit: 8c5d511
Built: Wed Sep 2 16:25:21 UTC 2015
OS/Arch: linux/amd64
Experimental: true

-bash-4.2# docker run -i -t --publish-service=svc2.dev.overlay busybox /bin/sh
/ # nslookup www.google.com
Server: 172.31.0.2
Address 1: 172.31.0.2

nslookup: can't resolve 'www.google.com'
/ # ping www.google.com
[no response]

Ping another container works:
/ # ping svc3
PING svc3 (172.21.0.8): 56 data bytes
64 bytes from 172.21.0.8: seq=0 ttl=64 time=0.571 ms
^C
--- svc3 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 0.571/0.571/0.571 ms

For containers created using the bridge network, I am able to ping www.google.com with overlay network. Do I need to do anything special for using overlay network?

@sanimej

This comment has been minimized.

Contributor

sanimej commented Oct 13, 2015

@baopx In the older libnetwork experimental code for containers connected to a overlay network you have to pass -p option for external connectivity. With the latest 1.9 code containers on overlay networks are connected automatically to a bridge network for external connectivity. There are also network UI changes. Please check the new network commands (libnetwork is out of experimental status now)

https://github.com/docker/docker/tree/master/docs/reference/commandline

@baopx

This comment has been minimized.

baopx commented Oct 13, 2015

Thanks sanimej,

Is there documentation on how I should start docker daemon, kv consul, to enable overlay network with latest 1.9 code? similar to https://github.com/docker/libnetwork/blob/master/docs/overlay.md ?

In the latest 1.9 code, --kv-store=consul:localhost:8500 is no longer a valid flag.

@sanimej

This comment has been minimized.

Contributor

sanimej commented Oct 13, 2015

@baopx There are changes to the cluster config as well.. Please refer to this updated document..

https://github.com/docker/docker/blob/master/docs/userguide/dockernetworks.md

@baopx

This comment has been minimized.

baopx commented Oct 14, 2015

One more question,

If I need to ping the container by IP/name.network from docker host, is this possible?

Thanks,
Bao

@baopx

This comment has been minimized.

baopx commented Oct 14, 2015

With the instructions at https://github.com/docker/docker/blob/master/docs/userguide/dockernetworks.md, the containers I created can reach external address and containers' address within the same host. When containers are on different docker host, they cannot ping one another. (Although, DNS is resolve)

Started docker through systemd, as follow:
node1:
ExecStart=/usr/bin/docker daemon -H fd:// --cluster-store=consul://localhost:8500 --label=com.docker.network.driver.overlay.bind_interface=eth0

node2 & 3:
ExecStart=/usr/bin/docker daemon -H fd:// --cluster-store=consul://localhost:8500 --label=com.docker.network.driver.overlay.bind_interface=eth0 --label=com.docker.network.driver.overlay.neighbor_ip=<NODE1_IP_ADDRESS>

Start consul as follow:
node1:
consul agent -server -bootstrap -data-dir /tmp/consul -bind=ifconfig eth0 | grep "inet " | awk '{print $2}'

node2 & 3:
consul agent -data-dir /tmp/consul -bind=ifconfig eth0 | grep "inet " | awk '{print $2}'
consul join <NODE1_IP_ADDRESS>

Create network:
docker network create -d overlay porx

Starting container:
docker run --net porx -itd --name net2box1 busybox

  1. Within the container, /etc/hosts has all IP addresses of other containers in the network

  2. Containers have public access (ping www.google.com works)

  3. If containers are in the same host, they can ping each other.

  4. But, if containers are on different docker hosts, I can resolve DNS on the container, but no network connectivity.

/ # cat /etc/hosts
10.0.0.4 02b9a9adbcc8
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
10.0.0.3 foobar1
10.0.0.3 foobar1.porx
10.0.0.2 porx_container
10.0.0.2 porx_container.porx
10.0.0.6 foobar1a
10.0.0.6 foobar1a.porx
10.0.0.5 porx_container3
10.0.0.5 porx_container3.porx
10.0.0.7 jovial_darwin
10.0.0.7 jovial_darwin.porx
/ # ping jovial_darwin
PING jovial_darwin (10.0.0.7): 56 data bytes
^C
--- jovial_darwin ping statistics ---
2 packets transmitted, 0 packets received, 100% packet loss
/ # ping foobar1a.porx
PING foobar1a.porx (10.0.0.6): 56 data bytes
^C
--- foobar1a.porx ping statistics ---

/ # ifconfig
eth0 Link encap:Ethernet HWaddr 02:42:0A:00:00:04
inet addr:10.0.0.4 Bcast:0.0.0.0 Mask:255.255.255.0
inet6 addr: fe80::42:aff:fe00:4/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1450 Metric:1
RX packets:15 errors:0 dropped:0 overruns:0 frame:0
TX packets:23 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1206 (1.1 KiB) TX bytes:1278 (1.2 KiB)

eth1 Link encap:Ethernet HWaddr 02:42:AC:12:00:03
inet addr:172.18.0.3 Bcast:0.0.0.0 Mask:255.255.0.0
inet6 addr: fe80::42:acff:fe12:3/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:14 errors:0 dropped:0 overruns:0 frame:0
TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1090 (1.0 KiB) TX bytes:982 (982.0 B)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:65536 Metric:1
RX packets:14 errors:0 dropped:0 overruns:0 frame:0
TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1568 (1.5 KiB) TX bytes:1568 (1.5 KiB)

/ # route
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
default ip-172-18-0-1.u 0.0.0.0 UG 0 0 0 eth1
10.0.0.0 * 255.255.255.0 U 0 0 0 eth0
172.18.0.0 * 255.255.0.0 U 0 0 0 eth1

@baopx baopx changed the title from public network not reachable on multi-host overlay network to network not reachable on multi-host overlay network Oct 14, 2015

@sanimej

This comment has been minimized.

Contributor

sanimej commented Oct 14, 2015

@baopx There are some changes to the UX for cluster configs. labels for bind_interface and neighbor_ip are no longer required. But in addition to --cluster-store you have to configure --cluster-advertise as well.. It is the local IP of the interface that connects to the cluster..

--cluster-store=consul://192.168.33.10:8500 --cluster-advertise=192.168.33.12:0

@kingbirdzheng

This comment has been minimized.

kingbirdzheng commented Nov 2, 2015

@sanimej would you please elaborate the cluster-advertise a bit more?
What should the IP address and Port for this label? Thanks.

@sanimej

This comment has been minimized.

Contributor

sanimej commented Nov 2, 2015

@kingbirdzheng In the config I gave earlier..

--cluster-store=consul://192.168.33.10:8500 --cluster-advertise=192.168.33.12:0

192.168.33.10 is the address of the node running the consul server. I am using virtualbox. All the nodes participating in the cluster are connected to the same network (using Host-only Adapter in my case). 192.168.33.12 is the IP of the local interface connected to that network.

@kingbirdzheng

This comment has been minimized.

kingbirdzheng commented Nov 3, 2015

@sanimej Thanks for your response.

What the port number 0 of 192.168.33.12 stands for?

@sanimej

This comment has been minimized.

Contributor

sanimej commented Nov 3, 2015

@kingbirdzheng port number is not used by the overlay driver.

@fcrisciani

This comment has been minimized.

Member

fcrisciani commented Aug 14, 2017

looks like there was no further follow up, closing

@fcrisciani fcrisciani closed this Aug 14, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment