New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

addvertise-wan only for inter-server communication? #1552

Closed
ChristianKniep opened this Issue Dec 28, 2015 · 11 comments

Comments

Projects
None yet
3 participants
@ChristianKniep

Hey there,

from my compose stack: https://github.com/ChristianKniep/orchestra/tree/master/consul-ambassador

I turns out that the advertise-wan address is only used in inter-server gossiping. Wouldn't it be nice if this IP is also advertised for external datacenter access? I elaborated on that in the README, which I just paste here... :)

Consul Ambassador

To fake some not-yet consularised services I would like to set-up a (some-what) 'hidden' DC that exposes services which are addvertised under the IP of the external service.

Why?

For starters 'Because I can', but on a more serious note - I want to containerise (and consulerise) a stack iteratively - one service at a time.

Until I haven't containerised service1 I am not able to use the service in consul-templates.

But if I could mock a hidden DC, which - once queried - exposes the IP of the external service, I could trick myself into heaven. :)

I would just have to specify { service "service1@int" } and it would expand with the external (aka advertised) IP.

How

We got two networks: [EDIT: If you do not have a docker-network ready cluster, scroll down - down there I use bridges instead of overlay.]

$ docker network create -d overlay global
# docker network create -d overlay --subnet=192.168.1.0/24 int

We got three internal consul agents:

  • int0: The central consul server, starting with the network global and connected to the network int afterwards.
  • int{1..2}: Two clients within the internal network, not reachable from the outside. int2
  • ext0: A consul agent connected to int0 as a WAN server.

First I start the internal server and attach the internal network afterwards:

$ docker-compose up -d int0
$ docker network connect int int0

The container now has two network interfaces:

$ docker exec -ti int0 ip -o -4 addr
1: lo    inet 127.0.0.1/8 scope host lo\       valid_lft forever preferred_lft forever
230: eth0    inet 10.0.0.2/24 scope global eth0\       valid_lft forever preferred_lft forever
232: eth1    inet 172.18.0.2/16 scope global eth1\       valid_lft forever preferred_lft forever
235: eth2    inet 192.168.1.2/24 scope global eth2\       valid_lft forever preferred_lft forever

The docker-compose.yml file might be adjusted to the correct IPs:

$ grep CONSUL_CLUSTER_IPS docker-compose.yml
   - CONSUL_CLUSTER_IPS=192.168.1.2
   - CONSUL_CLUSTER_IPS=192.168.1.2
$ grep WAN_SERVER docker-compose.yml
   - WAN_SERVER=10.0.0.2

Now start the rest:

$ docker-compose up -d
int0 is up-to-date
Creating ext0
Creating int1
Creating int2

Advertised Address

  • What I want: The client proposes the WAN address via DNS when asked from outside the DC.
  • What I got: I misunderstood the advertised address option - it seems only to play a role in the server2server gossip part. :/

Ping int0 (within the global network)

$ docker exec -ti ext0 ping -c1 int0.node.int.consul
PING int0.node.int.consul (10.0.0.2) 56(84) bytes of data.
64 bytes from int0 (10.0.0.2): icmp_seq=1 ttl=64 time=0.049 ms

Ping int1 (not reachable, since within the int network)

$ docker exec -ti ext0 ping -w1 -c1 int1.node.int.consul
PING int1.node.int.consul (192.168.1.3) 56(84) bytes of data.

--- int1.node.int.consul ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

Ping int2 (also not reachable, even though it advertises 8.8.8.8 )

$ docker exec -ti ext0 ping -w1 -c1 int2.node.int.consul
PING int2.node.int.consul (192.168.1.4) 56(84) bytes of data.

--- int2.node.int.consul ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

For this I would love the client to advertise his WAN address 8.8.8.8:

[root@int2 /]# cat /etc/consul.json
{
    "bootstrap": false,
    "server": false,
    "datacenter": "int",
    "data_dir": "/var/consul",
    "log_level": "INFO",
    "enable_syslog": false,
    "ui_dir": "/opt/consul-web-ui/",
    "client_addr": "0.0.0.0",
    "start_join": ["192.168.1.2"],
    "advertise_addr": "192.168.1.4",
    "advertise_addr_wan": "8.8.8.8",
    "node_name": "int2",
    "recursor": "8.8.8.8",
    "retry_max_wan": 3,
    "ports": {
        "dns": 53
    }
}
@slackpad

This comment has been minimized.

Show comment
Hide comment
@slackpad

slackpad Jan 13, 2016

Contributor

Hi @ChristianKniep there's a PR in review that I think is very similar to what you are proposing here. How does this look to you for your use case?

#1547

Contributor

slackpad commented Jan 13, 2016

Hi @ChristianKniep there's a PR in review that I think is very similar to what you are proposing here. How does this look to you for your use case?

#1547

@ChristianKniep

This comment has been minimized.

Show comment
Hide comment
@ChristianKniep

ChristianKniep Jan 13, 2016

I created an image with the patched version.

For some reason my version does not pick up the new configuration option "translate_wan_addrs": true,... :/

The Error

[root@int2 /]# cat /var/log/supervisor/consul.log
==> Error decoding '/etc/consul.json': Config has invalid keys: translate_wan_addrs
[root@int2 /]# cat etc/consul.json
{
    "bootstrap": false,
    "server": false,
    "datacenter": "int",
    "data_dir": "/var/consul",
    "log_level": "INFO",
    "enable_syslog": false,
    "ui_dir": "/opt/consul-web-ui/",
    "client_addr": "0.0.0.0",
    "start_join": ["192.168.1.2"],
    "advertise_addr": "192.168.1.3",
    "advertise_addr_wan": "8.8.8.8",
    "translate_wan_addrs": true,
    "node_name": "int2",
    "recursor": "8.8.8.8",
    "retry_max_wan": 3,
    "ports": {
        "dns": 53
    }
}
[root@int2 /]#

The Build

$ cat Dockerfile
FROM qnib/bats

RUN yum install -y unzip jq bc make golang git-core mercurial
# consul
ENV CONSUL_VER=0.6.0 \
    GOPATH=/usr/local/
RUN curl -fsL https://github.com/evan2645/consul/archive/add-wan-address-to-node.zip | bsdtar xf - -C /opt/ && \
    cd /opt/consul-add-wan-address-to-node/ && \
    go get -d && \
    go build -o /usr/local/bin/consul

The Test

I changed the consul-ambassador stack to use the image:

$ grep wan -B1 docker-compose.yml
int0:
  image: qnib/consul:wan
--
int1:
  image: qnib/consul:wan
--
int2:
  image: qnib/consul:wan
--
ext0:
  image: qnib/consul:wan

And replayed the test, this time on a single-node (using bridge, instead of overlay).

$ docker network create -d bridge --subnet=10.0.0.0/24 global
99ca3de93019e613e3ac17829838bec50587ff76723ba1e1f597f7f77d26b653
$ docker network create -d bridge --subnet=192.168.1.0/24 int
f92a571a6b29ad525286556c4381458d5a5c8800b09f030c398155b8f468c22a
$ docker-compose up -d int0
Creating int0
$ docker network connect int int0
$ docker exec -ti int0 ip -o -4 addr
1: lo    inet 127.0.0.1/8 scope host lo\       valid_lft forever preferred_lft forever
61: eth0    inet 10.0.0.2/24 scope global eth0\       valid_lft forever preferred_lft forever
63: eth1    inet 192.168.1.2/24 scope global eth1\       valid_lft forever preferred_lft forever
$ docker exec -ti int0 consul members
Node  Address        Status  Type    Build  Protocol  DC
int0  10.0.0.2:8301  alive   server  0.6.0  2         int
$ docker-compose up -d ext0
Creating ext0
$ docker exec -ti ext0 consul members
Node  Address        Status  Type    Build  Protocol  DC
ext0  10.0.0.3:8301  alive   server  0.6.0  2         ext
$ docker exec -ti ext0 consul members -wan
Node      Address        Status  Type    Build  Protocol  DC
ext0.ext  10.0.0.3:8302  alive   server  0.6.0  2         ext
int0.int  10.0.0.2:8302  alive   server  0.6.0  2         int
$ docker-compose up -d int2
Creating int2
$ docker exec -ti int2 bash
[root@int2 /]# vim /etc/consul.json

Add "translate_wan_addrs": true, underneath advertise_addr_wan, don't worry, the rest will be filled out by the startscript.

[root@int2 /]# grep -C2 trans /etc/consul.json
    "advertise_addr": "192.168.1.3",
    "advertise_addr_wan": "8.8.8.8",
    "translate_wan_addrs": true,
    "node_name": "int2",
    "recursor": "8.8.8.8",
[root@int2 /]# supervisorctl start consul
consul: ERROR (abnormal termination)
[root@int2 /]#

Error see above... :(

Missed, since the option was not picked up...

Now I'll try to ping int0 from ext0 which will use the wan interface, known to ext0:

$ docker exec -ti ext0 consul members -wan
Node      Address        Status  Type    Build  Protocol  DC
ext0.ext  10.0.0.3:8302  alive   server  0.6.0  2         ext
int0.int  10.0.0.2:8302  alive   server  0.6.0  2         int
$ docker exec -ti ext0 ping -c1 int0.node.int.consul
PING int0.node.int.consul (10.0.0.2) 56(84) bytes of data.
64 bytes from int0 (10.0.0.2): icmp_seq=1 ttl=64 time=0.217 ms

--- int0.node.int.consul ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.217/0.217/0.217/0.000 ms
$

An node hiden within the int DC is still not presented with the advertised wan adress.

$ docker exec -ti ext0 ping -w1 -c1 int2.node.int.consul
PING int2.node.int.consul (192.168.1.4) 56(84) bytes of data.

--- int2.node.int.consul ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms

I created an image with the patched version.

For some reason my version does not pick up the new configuration option "translate_wan_addrs": true,... :/

The Error

[root@int2 /]# cat /var/log/supervisor/consul.log
==> Error decoding '/etc/consul.json': Config has invalid keys: translate_wan_addrs
[root@int2 /]# cat etc/consul.json
{
    "bootstrap": false,
    "server": false,
    "datacenter": "int",
    "data_dir": "/var/consul",
    "log_level": "INFO",
    "enable_syslog": false,
    "ui_dir": "/opt/consul-web-ui/",
    "client_addr": "0.0.0.0",
    "start_join": ["192.168.1.2"],
    "advertise_addr": "192.168.1.3",
    "advertise_addr_wan": "8.8.8.8",
    "translate_wan_addrs": true,
    "node_name": "int2",
    "recursor": "8.8.8.8",
    "retry_max_wan": 3,
    "ports": {
        "dns": 53
    }
}
[root@int2 /]#

The Build

$ cat Dockerfile
FROM qnib/bats

RUN yum install -y unzip jq bc make golang git-core mercurial
# consul
ENV CONSUL_VER=0.6.0 \
    GOPATH=/usr/local/
RUN curl -fsL https://github.com/evan2645/consul/archive/add-wan-address-to-node.zip | bsdtar xf - -C /opt/ && \
    cd /opt/consul-add-wan-address-to-node/ && \
    go get -d && \
    go build -o /usr/local/bin/consul

The Test

I changed the consul-ambassador stack to use the image:

$ grep wan -B1 docker-compose.yml
int0:
  image: qnib/consul:wan
--
int1:
  image: qnib/consul:wan
--
int2:
  image: qnib/consul:wan
--
ext0:
  image: qnib/consul:wan

And replayed the test, this time on a single-node (using bridge, instead of overlay).

$ docker network create -d bridge --subnet=10.0.0.0/24 global
99ca3de93019e613e3ac17829838bec50587ff76723ba1e1f597f7f77d26b653
$ docker network create -d bridge --subnet=192.168.1.0/24 int
f92a571a6b29ad525286556c4381458d5a5c8800b09f030c398155b8f468c22a
$ docker-compose up -d int0
Creating int0
$ docker network connect int int0
$ docker exec -ti int0 ip -o -4 addr
1: lo    inet 127.0.0.1/8 scope host lo\       valid_lft forever preferred_lft forever
61: eth0    inet 10.0.0.2/24 scope global eth0\       valid_lft forever preferred_lft forever
63: eth1    inet 192.168.1.2/24 scope global eth1\       valid_lft forever preferred_lft forever
$ docker exec -ti int0 consul members
Node  Address        Status  Type    Build  Protocol  DC
int0  10.0.0.2:8301  alive   server  0.6.0  2         int
$ docker-compose up -d ext0
Creating ext0
$ docker exec -ti ext0 consul members
Node  Address        Status  Type    Build  Protocol  DC
ext0  10.0.0.3:8301  alive   server  0.6.0  2         ext
$ docker exec -ti ext0 consul members -wan
Node      Address        Status  Type    Build  Protocol  DC
ext0.ext  10.0.0.3:8302  alive   server  0.6.0  2         ext
int0.int  10.0.0.2:8302  alive   server  0.6.0  2         int
$ docker-compose up -d int2
Creating int2
$ docker exec -ti int2 bash
[root@int2 /]# vim /etc/consul.json

Add "translate_wan_addrs": true, underneath advertise_addr_wan, don't worry, the rest will be filled out by the startscript.

[root@int2 /]# grep -C2 trans /etc/consul.json
    "advertise_addr": "192.168.1.3",
    "advertise_addr_wan": "8.8.8.8",
    "translate_wan_addrs": true,
    "node_name": "int2",
    "recursor": "8.8.8.8",
[root@int2 /]# supervisorctl start consul
consul: ERROR (abnormal termination)
[root@int2 /]#

Error see above... :(

Missed, since the option was not picked up...

Now I'll try to ping int0 from ext0 which will use the wan interface, known to ext0:

$ docker exec -ti ext0 consul members -wan
Node      Address        Status  Type    Build  Protocol  DC
ext0.ext  10.0.0.3:8302  alive   server  0.6.0  2         ext
int0.int  10.0.0.2:8302  alive   server  0.6.0  2         int
$ docker exec -ti ext0 ping -c1 int0.node.int.consul
PING int0.node.int.consul (10.0.0.2) 56(84) bytes of data.
64 bytes from int0 (10.0.0.2): icmp_seq=1 ttl=64 time=0.217 ms

--- int0.node.int.consul ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.217/0.217/0.217/0.000 ms
$

An node hiden within the int DC is still not presented with the advertised wan adress.

$ docker exec -ti ext0 ping -w1 -c1 int2.node.int.consul
PING int2.node.int.consul (192.168.1.4) 56(84) bytes of data.

--- int2.node.int.consul ping statistics ---
2 packets transmitted, 0 received, 100% packet loss, time 999ms
@ChristianKniep

This comment has been minimized.

Show comment
Hide comment
@ChristianKniep

ChristianKniep Jan 13, 2016

Build like this fails, that's why I use go get -d and go build.

$ make
--> Installing build dependencies
github.com/mitchellh/gox (download)
*snip*
github.com/mitchellh/mapstructure (download)
github.com/ryanuber/columnize (download)
can't load package: package _/opt/consul-wan: cannot find package "_/opt/consul-wan" in any of:
    /usr/lib/golang/src/_/opt/consul-wan (from $GOROOT)
    /usr/local/src/_/opt/consul-wan (from $GOPATH)
can't load package: package _/opt/consul-wan/acl: cannot find package "_/opt/consul-wan/acl" in any of:
    /usr/lib/golang/src/_/opt/consul-wan/acl (from $GOROOT)
    /usr/local/src/_/opt/consul-wan/acl (from $GOPATH)
can't load package: package _/opt/consul-wan/api: cannot find package "_/opt/consul-wan/api" in any of:
    /usr/lib/golang/src/_/opt/consul-wan/api (from $GOROOT)
    /usr/local/src/_/opt/consul-wan/api (from $GOPATH)
can't load package: package _/opt/consul-wan/command: cannot find package "_/opt/consul-wan/command" in any of:
    /usr/lib/golang/src/_/opt/consul-wan/command (from $GOROOT)
    /usr/local/src/_/opt/consul-wan/command (from $GOPATH)
can't load package: package _/opt/consul-wan/command/agent: cannot find package "_/opt/consul-wan/command/agent" in any of:
    /usr/lib/golang/src/_/opt/consul-wan/command/agent (from $GOROOT)
    /usr/local/src/_/opt/consul-wan/command/agent (from $GOPATH)
can't load package: package _/opt/consul-wan/consul: cannot find package "_/opt/consul-wan/consul" in any of:
    /usr/lib/golang/src/_/opt/consul-wan/consul (from $GOROOT)
    /usr/local/src/_/opt/consul-wan/consul (from $GOPATH)
can't load package: package _/opt/consul-wan/consul/state: cannot find package "_/opt/consul-wan/consul/state" in any of:
    /usr/lib/golang/src/_/opt/consul-wan/consul/state (from $GOROOT)
    /usr/local/src/_/opt/consul-wan/consul/state (from $GOPATH)
can't load package: package _/opt/consul-wan/consul/structs: cannot find package "_/opt/consul-wan/consul/structs" in any of:
    /usr/lib/golang/src/_/opt/consul-wan/consul/structs (from $GOROOT)
    /usr/local/src/_/opt/consul-wan/consul/structs (from $GOPATH)
can't load package: package _/opt/consul-wan/testutil: cannot find package "_/opt/consul-wan/testutil" in any of:
    /usr/lib/golang/src/_/opt/consul-wan/testutil (from $GOROOT)
    /usr/local/src/_/opt/consul-wan/testutil (from $GOPATH)
can't load package: package _/opt/consul-wan/tlsutil: cannot find package "_/opt/consul-wan/tlsutil" in any of:
    /usr/lib/golang/src/_/opt/consul-wan/tlsutil (from $GOROOT)
    /usr/local/src/_/opt/consul-wan/tlsutil (from $GOPATH)
can't load package: package _/opt/consul-wan/watch: cannot find package "_/opt/consul-wan/watch" in any of:
    /usr/lib/golang/src/_/opt/consul-wan/watch (from $GOROOT)
    /usr/local/src/_/opt/consul-wan/watch (from $GOPATH)
make: *** [format] Error 1

Build like this fails, that's why I use go get -d and go build.

$ make
--> Installing build dependencies
github.com/mitchellh/gox (download)
*snip*
github.com/mitchellh/mapstructure (download)
github.com/ryanuber/columnize (download)
can't load package: package _/opt/consul-wan: cannot find package "_/opt/consul-wan" in any of:
    /usr/lib/golang/src/_/opt/consul-wan (from $GOROOT)
    /usr/local/src/_/opt/consul-wan (from $GOPATH)
can't load package: package _/opt/consul-wan/acl: cannot find package "_/opt/consul-wan/acl" in any of:
    /usr/lib/golang/src/_/opt/consul-wan/acl (from $GOROOT)
    /usr/local/src/_/opt/consul-wan/acl (from $GOPATH)
can't load package: package _/opt/consul-wan/api: cannot find package "_/opt/consul-wan/api" in any of:
    /usr/lib/golang/src/_/opt/consul-wan/api (from $GOROOT)
    /usr/local/src/_/opt/consul-wan/api (from $GOPATH)
can't load package: package _/opt/consul-wan/command: cannot find package "_/opt/consul-wan/command" in any of:
    /usr/lib/golang/src/_/opt/consul-wan/command (from $GOROOT)
    /usr/local/src/_/opt/consul-wan/command (from $GOPATH)
can't load package: package _/opt/consul-wan/command/agent: cannot find package "_/opt/consul-wan/command/agent" in any of:
    /usr/lib/golang/src/_/opt/consul-wan/command/agent (from $GOROOT)
    /usr/local/src/_/opt/consul-wan/command/agent (from $GOPATH)
can't load package: package _/opt/consul-wan/consul: cannot find package "_/opt/consul-wan/consul" in any of:
    /usr/lib/golang/src/_/opt/consul-wan/consul (from $GOROOT)
    /usr/local/src/_/opt/consul-wan/consul (from $GOPATH)
can't load package: package _/opt/consul-wan/consul/state: cannot find package "_/opt/consul-wan/consul/state" in any of:
    /usr/lib/golang/src/_/opt/consul-wan/consul/state (from $GOROOT)
    /usr/local/src/_/opt/consul-wan/consul/state (from $GOPATH)
can't load package: package _/opt/consul-wan/consul/structs: cannot find package "_/opt/consul-wan/consul/structs" in any of:
    /usr/lib/golang/src/_/opt/consul-wan/consul/structs (from $GOROOT)
    /usr/local/src/_/opt/consul-wan/consul/structs (from $GOPATH)
can't load package: package _/opt/consul-wan/testutil: cannot find package "_/opt/consul-wan/testutil" in any of:
    /usr/lib/golang/src/_/opt/consul-wan/testutil (from $GOROOT)
    /usr/local/src/_/opt/consul-wan/testutil (from $GOPATH)
can't load package: package _/opt/consul-wan/tlsutil: cannot find package "_/opt/consul-wan/tlsutil" in any of:
    /usr/lib/golang/src/_/opt/consul-wan/tlsutil (from $GOROOT)
    /usr/local/src/_/opt/consul-wan/tlsutil (from $GOPATH)
can't load package: package _/opt/consul-wan/watch: cannot find package "_/opt/consul-wan/watch" in any of:
    /usr/lib/golang/src/_/opt/consul-wan/watch (from $GOROOT)
    /usr/local/src/_/opt/consul-wan/watch (from $GOPATH)
make: *** [format] Error 1
@evan2645

This comment has been minimized.

Show comment
Hide comment
@evan2645

evan2645 Jan 13, 2016

Contributor

I'm no expert, but if I had to guess, it could be due to different project path and GOPATH...? The package names shouldn't be prefixed with _/opt/consul-wan and should instead just be consul.

I updated all my dependencies and built again cleanly, just to be sure

Contributor

evan2645 commented Jan 13, 2016

I'm no expert, but if I had to guess, it could be due to different project path and GOPATH...? The package names shouldn't be prefixed with _/opt/consul-wan and should instead just be consul.

I updated all my dependencies and built again cleanly, just to be sure

@ChristianKniep

This comment has been minimized.

Show comment
Hide comment
@ChristianKniep

ChristianKniep Jan 15, 2016

Hey Evan,
I am a bit confused about the GOPATH setting. I tried to cheat my way forward by symlinking the path, but the gods were not in favour...

[root@f0b4f5753033 /]# mkdir -p /usr/lib/golang/src/_/opt/
[root@f0b4f5753033 /]# git clone https://github.com/evan2645/consul.git /opt/consul/
Cloning into '/opt/consul'...
*snip*
Checking connectivity... done.
[root@f0b4f5753033 /]# ln -s /opt/consul /usr/lib/golang/src/_/opt/
[root@f0b4f5753033 /]# cd /opt/consul/
[root@f0b4f5753033 consul]# make
--> Installing build dependencies
*snip*
github.com/ryanuber/columnize (download)
--> Running go fmt
==> Getting dependencies...
go install: no install location for directory /opt/consul outside GOPATH
go install: no install location for directory /opt/consul/acl outside GOPATH
go install: no install location for directory /opt/consul/api outside GOPATH
go install: no install location for directory /opt/consul/command outside GOPATH
go install: no install location for directory /opt/consul/command/agent outside GOPATH
go install: no install location for directory /opt/consul/consul outside GOPATH
go install: no install location for directory /opt/consul/consul/state outside GOPATH
go install: no install location for directory /opt/consul/consul/structs outside GOPATH
go install: no install location for directory /opt/consul/testutil outside GOPATH
go install: no install location for directory /opt/consul/tlsutil outside GOPATH
go install: no install location for directory /opt/consul/watch outside GOPATH
make: *** [all] Error 1
[root@f0b4f5753033 consul]#

Hey Evan,
I am a bit confused about the GOPATH setting. I tried to cheat my way forward by symlinking the path, but the gods were not in favour...

[root@f0b4f5753033 /]# mkdir -p /usr/lib/golang/src/_/opt/
[root@f0b4f5753033 /]# git clone https://github.com/evan2645/consul.git /opt/consul/
Cloning into '/opt/consul'...
*snip*
Checking connectivity... done.
[root@f0b4f5753033 /]# ln -s /opt/consul /usr/lib/golang/src/_/opt/
[root@f0b4f5753033 /]# cd /opt/consul/
[root@f0b4f5753033 consul]# make
--> Installing build dependencies
*snip*
github.com/ryanuber/columnize (download)
--> Running go fmt
==> Getting dependencies...
go install: no install location for directory /opt/consul outside GOPATH
go install: no install location for directory /opt/consul/acl outside GOPATH
go install: no install location for directory /opt/consul/api outside GOPATH
go install: no install location for directory /opt/consul/command outside GOPATH
go install: no install location for directory /opt/consul/command/agent outside GOPATH
go install: no install location for directory /opt/consul/consul outside GOPATH
go install: no install location for directory /opt/consul/consul/state outside GOPATH
go install: no install location for directory /opt/consul/consul/structs outside GOPATH
go install: no install location for directory /opt/consul/testutil outside GOPATH
go install: no install location for directory /opt/consul/tlsutil outside GOPATH
go install: no install location for directory /opt/consul/watch outside GOPATH
make: *** [all] Error 1
[root@f0b4f5753033 consul]#
@ChristianKniep

This comment has been minimized.

Show comment
Hide comment
@ChristianKniep

ChristianKniep Jan 15, 2016

I got the same error in upstream-consul. :/ So it might be me who's the problem?

I got the same error in upstream-consul. :/ So it might be me who's the problem?

@ChristianKniep

This comment has been minimized.

Show comment
Hide comment
@ChristianKniep

ChristianKniep Jan 15, 2016

Might be the go version though.... Installing update...

[root@1b29de854621 /]# go version
go version go1.4.2 linux/amd64

Might be the go version though.... Installing update...

[root@1b29de854621 /]# go version
go version go1.4.2 linux/amd64
@ChristianKniep

This comment has been minimized.

Show comment
Hide comment
@ChristianKniep

ChristianKniep Jan 15, 2016

Nope, same error with 1.5:

Step 4 : RUN curl -sfL https://storage.googleapis.com/golang/go${GOVERSION}.linux-${ARCH}.tar.gz |bsdtar xf - -C /opt/ &&     ln -s /opt/go/bin/go /usr/local/bin/ &&     go version
 ---> Running in 326a9931ef0c
go version go1.5.3 linux/amd64
 ---> 6b00cbe15fc9
Removing intermediate container 326a9931ef0c
Step 5 : RUN git clone https://github.com/evan2645/consul.git /opt/consul/ &&     cd /opt/consul/ &&     git checkout add-wan-address-to-node &&     make
 ---> Running in eaa47a65f4f4
Cloning into '/opt/consul'...
Switched to a new branch 'add-wan-address-to-node'
Branch add-wan-address-to-node set up to track remote branch add-wan-address-to-node from origin.
--> Installing build dependencies
github.com/mitchellh/gox (download)
*snip*
github.com/ryanuber/columnize (download)
--> Running go fmt
can't load package: package _/opt/consul: cannot find package "_/opt/consul" in any of:
    /opt/go/src/_/opt/consul (from $GOROOT)
    /usr/local/src/_/opt/consul (from $GOPATH)
can't load package: package _/opt/consul/acl: cannot find package "_/opt/consul/acl" in any of:
    /opt/go/src/_/opt/consul/acl (from $GOROOT)
    /usr/local/src/_/opt/consul/acl (from $GOPATH)
can't load package: package _/opt/consul/api: cannot find package "_/opt/consul/api" in any of:
    /opt/go/src/_/opt/consul/api (from $GOROOT)
    /usr/local/src/_/opt/consul/api (from $GOPATH)
can't load package: package _/opt/consul/command: cannot find package "_/opt/consul/command" in any of:
    /opt/go/src/_/opt/consul/command (from $GOROOT)
    /usr/local/src/_/opt/consul/command (from $GOPATH)
can't load package: package _/opt/consul/command/agent: cannot find package "_/opt/consul/command/agent" in any of:
    /opt/go/src/_/opt/consul/command/agent (from $GOROOT)
    /usr/local/src/_/opt/consul/command/agent (from $GOPATH)
can't load package: package _/opt/consul/consul: cannot find package "_/opt/consul/consul" in any of:
    /opt/go/src/_/opt/consul/consul (from $GOROOT)
    /usr/local/src/_/opt/consul/consul (from $GOPATH)
can't load package: package _/opt/consul/consul/state: cannot find package "_/opt/consul/consul/state" in any of:
    /opt/go/src/_/opt/consul/consul/state (from $GOROOT)
    /usr/local/src/_/opt/consul/consul/state (from $GOPATH)
can't load package: package _/opt/consul/consul/structs: cannot find package "_/opt/consul/consul/structs" in any of:
    /opt/go/src/_/opt/consul/consul/structs (from $GOROOT)
    /usr/local/src/_/opt/consul/consul/structs (from $GOPATH)
can't load package: package _/opt/consul/testutil: cannot find package "_/opt/consul/testutil" in any of:
    /opt/go/src/_/opt/consul/testutil (from $GOROOT)
    /usr/local/src/_/opt/consul/testutil (from $GOPATH)
can't load package: package _/opt/consul/tlsutil: cannot find package "_/opt/consul/tlsutil" in any of:
    /opt/go/src/_/opt/consul/tlsutil (from $GOROOT)
    /usr/local/src/_/opt/consul/tlsutil (from $GOPATH)
can't load package: package _/opt/consul/watch: cannot find package "_/opt/consul/watch" in any of:
    /opt/go/src/_/opt/consul/watch (from $GOROOT)
    /usr/local/src/_/opt/consul/watch (from $GOPATH)
make: *** [format] Error 1
The command '/bin/sh -c git clone https://github.com/evan2645/consul.git /opt/consul/ &&     cd /opt/consul/ &&     git checkout add-wan-address-to-node &&     make' returned a non-zero code: 2

Nope, same error with 1.5:

Step 4 : RUN curl -sfL https://storage.googleapis.com/golang/go${GOVERSION}.linux-${ARCH}.tar.gz |bsdtar xf - -C /opt/ &&     ln -s /opt/go/bin/go /usr/local/bin/ &&     go version
 ---> Running in 326a9931ef0c
go version go1.5.3 linux/amd64
 ---> 6b00cbe15fc9
Removing intermediate container 326a9931ef0c
Step 5 : RUN git clone https://github.com/evan2645/consul.git /opt/consul/ &&     cd /opt/consul/ &&     git checkout add-wan-address-to-node &&     make
 ---> Running in eaa47a65f4f4
Cloning into '/opt/consul'...
Switched to a new branch 'add-wan-address-to-node'
Branch add-wan-address-to-node set up to track remote branch add-wan-address-to-node from origin.
--> Installing build dependencies
github.com/mitchellh/gox (download)
*snip*
github.com/ryanuber/columnize (download)
--> Running go fmt
can't load package: package _/opt/consul: cannot find package "_/opt/consul" in any of:
    /opt/go/src/_/opt/consul (from $GOROOT)
    /usr/local/src/_/opt/consul (from $GOPATH)
can't load package: package _/opt/consul/acl: cannot find package "_/opt/consul/acl" in any of:
    /opt/go/src/_/opt/consul/acl (from $GOROOT)
    /usr/local/src/_/opt/consul/acl (from $GOPATH)
can't load package: package _/opt/consul/api: cannot find package "_/opt/consul/api" in any of:
    /opt/go/src/_/opt/consul/api (from $GOROOT)
    /usr/local/src/_/opt/consul/api (from $GOPATH)
can't load package: package _/opt/consul/command: cannot find package "_/opt/consul/command" in any of:
    /opt/go/src/_/opt/consul/command (from $GOROOT)
    /usr/local/src/_/opt/consul/command (from $GOPATH)
can't load package: package _/opt/consul/command/agent: cannot find package "_/opt/consul/command/agent" in any of:
    /opt/go/src/_/opt/consul/command/agent (from $GOROOT)
    /usr/local/src/_/opt/consul/command/agent (from $GOPATH)
can't load package: package _/opt/consul/consul: cannot find package "_/opt/consul/consul" in any of:
    /opt/go/src/_/opt/consul/consul (from $GOROOT)
    /usr/local/src/_/opt/consul/consul (from $GOPATH)
can't load package: package _/opt/consul/consul/state: cannot find package "_/opt/consul/consul/state" in any of:
    /opt/go/src/_/opt/consul/consul/state (from $GOROOT)
    /usr/local/src/_/opt/consul/consul/state (from $GOPATH)
can't load package: package _/opt/consul/consul/structs: cannot find package "_/opt/consul/consul/structs" in any of:
    /opt/go/src/_/opt/consul/consul/structs (from $GOROOT)
    /usr/local/src/_/opt/consul/consul/structs (from $GOPATH)
can't load package: package _/opt/consul/testutil: cannot find package "_/opt/consul/testutil" in any of:
    /opt/go/src/_/opt/consul/testutil (from $GOROOT)
    /usr/local/src/_/opt/consul/testutil (from $GOPATH)
can't load package: package _/opt/consul/tlsutil: cannot find package "_/opt/consul/tlsutil" in any of:
    /opt/go/src/_/opt/consul/tlsutil (from $GOROOT)
    /usr/local/src/_/opt/consul/tlsutil (from $GOPATH)
can't load package: package _/opt/consul/watch: cannot find package "_/opt/consul/watch" in any of:
    /opt/go/src/_/opt/consul/watch (from $GOROOT)
    /usr/local/src/_/opt/consul/watch (from $GOPATH)
make: *** [format] Error 1
The command '/bin/sh -c git clone https://github.com/evan2645/consul.git /opt/consul/ &&     cd /opt/consul/ &&     git checkout add-wan-address-to-node &&     make' returned a non-zero code: 2
@slackpad

This comment has been minimized.

Show comment
Hide comment
@slackpad

slackpad Jan 15, 2016

Contributor

@ChristianKniep I've never gotten a symlink to work properly with the go build tools. I know that cloning your dev repo and actually placing it at github.com/hashicorp/consul in your GOPATH will work, even though it's a little gross to have to do that.

Contributor

slackpad commented Jan 15, 2016

@ChristianKniep I've never gotten a symlink to work properly with the go build tools. I know that cloning your dev repo and actually placing it at github.com/hashicorp/consul in your GOPATH will work, even though it's a little gross to have to do that.

@ChristianKniep

This comment has been minimized.

Show comment
Hide comment
@ChristianKniep

ChristianKniep Jan 16, 2016

OK, my bad - I didn't quite obey the GOPATH stuff... :) Now I got it...

It works! dancing
The members:

[root@int0 /]# consul members
Node  Address           Status  Type    Build  Protocol  DC
int0  10.0.0.2:8301     alive   server  0.6.0  2         int
int2  192.168.1.3:8301  alive   client  0.6.0  2         int
[root@int0 /]# consul members -wan
Node      Address        Status  Type    Build  Protocol  DC
ext0.ext  10.0.0.3:8302  alive   server  0.6.0  2         ext
int0.int  10.0.0.2:8302  alive   server  0.6.0  2         int
[root@int0 /]#

Pinging int2 from ext0 uses the WAN address of the client. :)

[root@ext0 /]# ping -c1 int2.node.int.consul
PING int2.node.int.consul (8.8.8.8) 56(84) bytes of data.
64 bytes from google-public-dns-a.google.com (8.8.8.8): icmp_seq=1 ttl=61 time=48.5 ms

--- int2.node.int.consul ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 48.599/48.599/48.599/0.000 ms
[root@ext0 /]#

But that's a thing of the DNS server, thus of the agent of the host (container) which wants to reach out.
Which is fine, I was expecting to leave the disicion to the agent within the service that should be reached.

Anyway - that's fine to me. I'll write up a blog post and publish it here: http://qnib.org

OK, my bad - I didn't quite obey the GOPATH stuff... :) Now I got it...

It works! dancing
The members:

[root@int0 /]# consul members
Node  Address           Status  Type    Build  Protocol  DC
int0  10.0.0.2:8301     alive   server  0.6.0  2         int
int2  192.168.1.3:8301  alive   client  0.6.0  2         int
[root@int0 /]# consul members -wan
Node      Address        Status  Type    Build  Protocol  DC
ext0.ext  10.0.0.3:8302  alive   server  0.6.0  2         ext
int0.int  10.0.0.2:8302  alive   server  0.6.0  2         int
[root@int0 /]#

Pinging int2 from ext0 uses the WAN address of the client. :)

[root@ext0 /]# ping -c1 int2.node.int.consul
PING int2.node.int.consul (8.8.8.8) 56(84) bytes of data.
64 bytes from google-public-dns-a.google.com (8.8.8.8): icmp_seq=1 ttl=61 time=48.5 ms

--- int2.node.int.consul ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 48.599/48.599/48.599/0.000 ms
[root@ext0 /]#

But that's a thing of the DNS server, thus of the agent of the host (container) which wants to reach out.
Which is fine, I was expecting to leave the disicion to the agent within the service that should be reached.

Anyway - that's fine to me. I'll write up a blog post and publish it here: http://qnib.org

@ChristianKniep

This comment has been minimized.

Show comment
Hide comment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment