Can't connect remotely to a server #16

jbrodriguez · 2015-11-04T16:25:42Z

Hi

I have a client running on 'blackbeard' machine and a server running on 'wopr' machine.

The consul server is running on blackbeard and a consul agent is running on wopr.

This is the consul server

⟩ env GOMAXPROCS=4 consul agent -server -bootstrap-expect 1 -data-dir /tmp/consul
==> WARNING: BootstrapExpect Mode is specified as 1; this is the same as Bootstrap mode.
==> WARNING: Bootstrap mode enabled! Do not enable unless necessary
==> Starting raft data migration...
==> Starting Consul agent...
==> Starting Consul agent RPC...
==> Consul agent running!
         Node name: 'blackbeard'
        Datacenter: 'dc1'
            Server: true (bootstrap: true)
       Client Addr: 127.0.0.1 (HTTP: 8500, HTTPS: -1, DNS: 8600, RPC: 8400)
      Cluster Addr: 192.168.23.23 (LAN: 8301, WAN: 8302)
    Gossip encrypt: false, RPC-TLS: false, TLS-Incoming: false
             Atlas: <disabled>

==> Log data will now stream in as it occurs:

    2015/11/04 10:50:08 [INFO] raft: Node at 192.168.23.23:8300 [Follower] entering Follower state
    2015/11/04 10:50:08 [INFO] serf: EventMemberJoin: blackbeard 192.168.23.23
    2015/11/04 10:50:08 [INFO] consul: adding server blackbeard (Addr: 192.168.23.23:8300) (DC: dc1)
    2015/11/04 10:50:08 [INFO] serf: EventMemberJoin: blackbeard.dc1 192.168.23.23
    2015/11/04 10:50:08 [INFO] consul: adding server blackbeard.dc1 (Addr: 192.168.23.23:8300) (DC: dc1)
    2015/11/04 10:50:08 [ERR] agent: failed to sync remote state: No cluster leader
    2015/11/04 10:50:09 [WARN] raft: Heartbeat timeout reached, starting election
    2015/11/04 10:50:09 [INFO] raft: Node at 192.168.23.23:8300 [Candidate] entering Candidate state
    2015/11/04 10:50:09 [INFO] raft: Election won. Tally: 1
    2015/11/04 10:50:09 [INFO] raft: Node at 192.168.23.23:8300 [Leader] entering Leader state
    2015/11/04 10:50:09 [INFO] consul: cluster leadership acquired
    2015/11/04 10:50:09 [INFO] consul: New leader elected: blackbeard
    2015/11/04 10:50:09 [INFO] raft: Disabling EnableSingleNode (bootstrap)
    2015/11/04 10:54:47 [INFO] agent: Deregistered service 'io.jbrodriguez.mediagui.scanner-47b07d7d-827e-11e5-81dc-d05099660776'
    2015/11/04 10:56:59 [INFO] agent: Deregistered service 'io.jbrodriguez.mediagui.scanner-47b07d7d-827e-11e5-81dc-d05099660776'
    2015/11/04 11:02:29 [INFO] serf: EventMemberJoin: wopr 192.168.23.72
    2015/11/04 11:02:29 [INFO] consul: member 'wopr' joined, marking health alive
    2015/11/04 11:12:30 [ERR] yamux: Failed to write header: use of closed network connection

This is the consult agent (client)

root@wopr:~# /boot/consul agent -data-dir /tmp/consul -join blackbeard.apertoire.org
==> WARNING: It is highly recommended to set GOMAXPROCS higher than 1
==> Starting Consul agent...
==> Starting Consul agent RPC...
==> Joining cluster...
    Join completed. Synced with 1 initial agents
==> Consul agent running!
         Node name: 'wopr'
        Datacenter: 'dc1'
            Server: false (bootstrap: false)
       Client Addr: 127.0.0.1 (HTTP: 8500, HTTPS: -1, DNS: 8600, RPC: 8400)
      Cluster Addr: 192.168.23.72 (LAN: 8301, WAN: 8302)
    Gossip encrypt: false, RPC-TLS: false, TLS-Incoming: false
             Atlas: <disabled>

==> Log data will now stream in as it occurs:

    2015/11/04 11:02:29 [INFO] serf: EventMemberJoin: wopr 192.168.23.72
    2015/11/04 11:02:29 [INFO] agent: (LAN) joining: [blackbeard.apertoire.org]
    2015/11/04 11:02:29 [INFO] serf: EventMemberJoin: blackbeard 192.168.23.23
    2015/11/04 11:02:29 [INFO] agent: (LAN) joined: 1 Err: <nil>
    2015/11/04 11:02:29 [ERR] agent: failed to sync remote state: No known Consul servers
    2015/11/04 11:02:29 [INFO] consul: adding server blackbeard (Addr: 192.168.23.23:8300) (DC: dc1)

This is the server

root@wopr:~# /boot/scanner -logtostderr
I1104 11:05:08.097673   25766 scanner.go:18] after parse
I1104 11:05:08.097767   25766 server.go:96] Starting server io.jbrodriguez.mediagui.scanner id io.jbrodriguez.mediagui.scanner-d1b784a8-830d-11e5-a519-d05099660776
I1104 11:05:08.097850   25766 rpc_server.go:222] Listening on 0.0.0.0:58719
I1104 11:05:08.097909   25766 http_broker.go:80] Broker Listening on 0.0.0.0:57574
I1104 11:05:08.097968   25766 rpc_server.go:152] Registering node: io.jbrodriguez.mediagui.scanner-d1b784a8-830d-11e5-a519-d05099660776
I1104 11:10:09.004940   25766 handler.go:19] Received scanner.Scan request: %vfolders:"/mnt/user/films"
I1104 11:10:18.540523   25766 handler.go:19] Received scanner.Scan request: %vfolders:"/mnt/user/films"
I1104 11:10:31.820802   25766 handler.go:19] Received scanner.Scan request: %vfolders:"/mnt/user/films"

This is the output of http://localhost:8500/v1/catalog/service/io.jbrodriguez.mediagui.scanner, run on blackbeard

[
  {
    "Node": "io.jbrodriguez.mediagui.scanner-d1b784a8-830d-11e5-a519-d05099660776",
    "Address": "0.0.0.0",
    "ServiceID": "io.jbrodriguez.mediagui.scanner-d1b784a8-830d-11e5-a519-d05099660776",
    "ServiceName": "io.jbrodriguez.mediagui.scanner",
    "ServiceTags": [
      "e={\"Name\":\"Scanner.Scan\",\"Request\":{\"Name\":\"Request\",\"Type\":\"Request\",\"Values\":[{\"Name\":\"Folders\",\"Type\":\"\",\"Values\":null}]},\"Response\":{\"Name\":\"Response\",\"Type\":\"Response\",\"Values\":[{\"Name\":\"Filenames\",\"Type\":\"\",\"Values\":null}]},\"Metadata\":{\"stream\":\"false\"}}",
      "e={\"Name\":\"Debug.Health\",\"Request\":{\"Name\":\"Request\",\"Type\":\"Request\",\"Values\":null},\"Response\":{\"Name\":\"Response\",\"Type\":\"Response\",\"Values\":[{\"Name\":\"Status\",\"Type\":\"string\",\"Values\":null}]},\"Metadata\":{\"stream\":\"false\"}}"
    ],
    "ServiceAddress": "",
    "ServicePort": 58719
  }
]

But the client fails to connect to server

⟩ ./test -logtostderr
I1104 11:10:28.359626   25332 main.go:25] req=&{service:io.jbrodriguez.mediagui.scanner method:Scanner.Scan contentType:application/octet-stream request:0xc82000b160}
W1104 11:10:28.360803   25332 main.go:31] Unable to connect to scanning service: %s{"id":"go.micro.client","code":500,"detail":"Error sending request: dial tcp 0.0.0.0:58719: getsockopt: connection refused","status":"Internal Server Error"}

If I run the client on wopr, it works

The text was updated successfully, but these errors were encountered:

asim · 2015-11-04T16:50:48Z

Thanks for raising the issue. The way in which a service registers, it blindly uses the server listener address provided rather than figuring out the appropriate IP of the machine or allow you to specify and advertising address. The quick fix is to set server.Address("10.0.0.1:8080") to the private ip of the host. I'll put together a more elegant solution that will either allow the separation of listen and advertise address much like consul does. Any thoughts or PRs welcome.

jbrodriguez · 2015-11-04T17:29:13Z

Thank you @asim , the quick fix worked !

asim · 2015-11-04T21:54:51Z

I've committed a fix here 0dbe14a. When 0.0.0.0 is specified as the bind address it will extract the private ip and use that for registration. Please close the issue if to satisfaction.

jbrodriguez · 2015-11-04T22:06:13Z

Not sure if this is the way to do it, but it's not working here

    server.Init(
        server.Name("io.jbrodriguez.mediagui.scanner."+host),
        server.Address("0.0.0.0:0"),
    )

root@wopr:~# /boot/scanner -logtostderr
I1104 17:03:34.004338    7537 scanner.go:55] Scanner started
I1104 17:03:34.004415    7537 server.go:96] Starting server io.jbrodriguez.mediagui.scanner.wopr id io.jbrodriguez.mediagui.scanner.wopr-e43c7501-833f-11e5-9483-d05099660776
I1104 17:03:34.004576    7537 rpc_server.go:222] Listening on 0.0.0.0:39241
I1104 17:03:34.004666    7537 http_broker.go:80] Broker Listening on 0.0.0.0:43409
I1104 17:03:34.004720    7537 rpc_server.go:152] Registering node: io.jbrodriguez.mediagui.scanner.wopr-e43c7501-833f-11e5-9483-d05099660776

[
  {
    "Node": "io.jbrodriguez.mediagui.scanner.wopr-e43c7501-833f-11e5-9483-d05099660776",
    "Address": "0.0.0.0",
    "ServiceID": "io.jbrodriguez.mediagui.scanner.wopr-e43c7501-833f-11e5-9483-d05099660776",
    "ServiceName": "io.jbrodriguez.mediagui.scanner.wopr",
    "ServiceTags": [
      "e={\"Name\":\"Scanner.Scan\",\"Request\":{\"Name\":\"Request\",\"Type\":\"Request\",\"Values\":[{\"Name\":\"Folders\",\"Type\":\"\",\"Values\":null}]},\"Response\":{\"Name\":\"Response\",\"Type\":\"Response\",\"Values\":[{\"Name\":\"Filenames\",\"Type\":\"\",\"Values\":null}]},\"Metadata\":{\"stream\":\"false\"}}",
      "e={\"Name\":\"Debug.Health\",\"Request\":{\"Name\":\"Request\",\"Type\":\"Request\",\"Values\":null},\"Response\":{\"Name\":\"Response\",\"Type\":\"Response\",\"Values\":[{\"Name\":\"Status\",\"Type\":\"string\",\"Values\":null}]},\"Metadata\":{\"stream\":\"false\"}}"
    ],
    "ServiceAddress": "",
    "ServicePort": 39241
  }
]

jbrodriguez · 2015-11-04T22:10:11Z

I think this line will never evaluate to true

0dbe14a#diff-83b1f985ddf5acc292d6df4ede4b6395R143

asim · 2015-11-04T22:13:49Z

Does your machine have private ips?

jbrodriguez · 2015-11-04T22:14:25Z

Yes, 192.168.23.x

asim · 2015-11-04T22:17:51Z

The IPs are assigned on the machine and you can see them if you do ifconfig?

jbrodriguez · 2015-11-04T22:30:14Z

Yes, I have docker running but that's a different issue

root@wopr:~# ifconfig
docker0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.17.42.1  netmask 255.255.0.0  broadcast 0.0.0.0
        ether 36:dd:68:31:e1:d2  txqueuelen 0  (Ethernet)
        RX packets 0  bytes 0 (0.0 B)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 558  bytes 72965 (71.2 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.23.72  netmask 255.255.255.0  broadcast 192.168.23.255
        ether d0:50:99:66:07:76  txqueuelen 1000  (Ethernet)
        RX packets 438818  bytes 481549074 (459.2 MiB)
        RX errors 0  dropped 1817  overruns 0  frame 0
        TX packets 210916  bytes 107895564 (102.8 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0
        device interrupt 20  memory 0xefd00000-efd20000

lo: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        loop  txqueuelen 0  (Local Loopback)
        RX packets 2925  bytes 35495980 (33.8 MiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 2925  bytes 35495980 (33.8 MiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

I'm running your code inside the service handler and it works

    for _, b := range []string{"10.0.0.0/8", "172.16.0.0/12", "192.168.0.0/16"} {
        if _, block, err := net.ParseCIDR(b); err == nil {
            privateBlocks = append(privateBlocks, block)
        }
    }

    addrs, err := net.InterfaceAddrs()
    if err != nil {
        log.Info("Oops: " + err.Error() + "\n")
        os.Exit(1)
    }

    ip := "0.0.0.0"
    for _, a := range addrs {
        if ipnet, ok := a.(*net.IPNet); ok && !ipnet.IP.IsLoopback() {
            if ipnet.IP.To4() != nil {
                for _, priv := range privateBlocks {
                    tip := net.ParseIP(ipnet.IP.String())
                    if priv.Contains(tip) {
                        log.Info("inside")
                        ip = ipnet.IP.String()
                    }
                }
                // if strings.Contains(ipnet.IP.String(), "192.168.0.0/16") {
                //  ip = ipnet.IP.String()
            }
        }
    }

    log.Infof("Ip=%s", ip)

outputs

root@wopr:~# /boot/scanner -logtostderr
I1104 17:24:29.243671    3652 scanner.go:46] inside
I1104 17:24:29.243757    3652 scanner.go:46] inside
I1104 17:24:29.243780    3652 scanner.go:56] Ip=172.17.42.1
I1104 17:24:29.243826    3652 scanner.go:71] Scanner started
I1104 17:24:29.243847    3652 server.go:96] Starting server io.jbrodriguez.mediagui.scanner.wopr id io.jbrodriguez.mediagui.scanner.wopr-d06a9fb1-8342-11e5-b6a2-d05099660776
I1104 17:24:29.243953    3652 rpc_server.go:222] Listening on 0.0.0.0:38084
I1104 17:24:29.244032    3652 http_broker.go:80] Broker Listening on 0.0.0.0:47733
I1104 17:24:29.244103    3652 rpc_server.go:152] Registering node: io.jbrodriguez.mediagui.scanner.wopr-d06a9fb1-8342-11e5-b6a2-d05099660776
^CI1104 17:26:00.164628    3652 server.go:85] Received signal interrupt
I1104 17:26:00.164654    3652 rpc_server.go:196] Deregistering node: io.jbrodriguez.mediagui.scanner.wopr-d06a9fb1-8342-11e5-b6a2-d05099660776
I1104 17:26:00.166701    3652 server.go:101] Stopping server
root@wopr:~# /boot/scanner -logtostderr
I1104 17:26:14.362823    6063 scanner.go:46] inside
I1104 17:26:14.362917    6063 scanner.go:46] inside
I1104 17:26:14.362950    6063 scanner.go:56] Ip=172.17.42.1
I1104 17:26:14.363032    6063 scanner.go:71] Scanner started
I1104 17:26:14.363089    6063 server.go:96] Starting server io.jbrodriguez.mediagui.scanner.wopr id io.jbrodriguez.mediagui.scanner.wopr-0f129758-8343-11e5-b8a2-d05099660776
I1104 17:26:14.363253    6063 rpc_server.go:222] Listening on 0.0.0.0:46509
I1104 17:26:14.363343    6063 http_broker.go:80] Broker Listening on 0.0.0.0:48304
I1104 17:26:14.363398    6063 rpc_server.go:152] Registering node: io.jbrodriguez.mediagui.scanner.wopr-0f129758-8343-11e5-b8a2-d05099660776
^CI1104 17:26:22.837228    6063 server.go:85] Received signal interrupt
I1104 17:26:22.837351    6063 rpc_server.go:196] Deregistering node: io.jbrodriguez.mediagui.scanner.wopr-0f129758-8343-11e5-b8a2-d05099660776
I1104 17:26:22.840463    6063 server.go:101] Stopping server

asim · 2015-11-04T22:31:26Z

Did you do go get -u? Perhaps the library was not rebuilt the first time. The behaviour suggests it wasn't updated.

jbrodriguez · 2015-11-04T22:36:16Z

That was it.

Thanks @asim !!

…ption file store: add DirOption

jbrodriguez closed this as completed Nov 4, 2015

Ak-Army pushed a commit to Ak-Army/go-micro that referenced this issue Jul 23, 2024

Merge pull request micro#16 from butonic/file-store-add-default-dir-o…

fe827c4

…ption file store: add DirOption

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can't connect remotely to a server #16

Can't connect remotely to a server #16

jbrodriguez commented Nov 4, 2015

asim commented Nov 4, 2015

jbrodriguez commented Nov 4, 2015

asim commented Nov 4, 2015

jbrodriguez commented Nov 4, 2015

jbrodriguez commented Nov 4, 2015

asim commented Nov 4, 2015

jbrodriguez commented Nov 4, 2015

asim commented Nov 4, 2015

jbrodriguez commented Nov 4, 2015

asim commented Nov 4, 2015

jbrodriguez commented Nov 4, 2015

Can't connect remotely to a server #16

Can't connect remotely to a server #16

Comments

jbrodriguez commented Nov 4, 2015

asim commented Nov 4, 2015

jbrodriguez commented Nov 4, 2015

asim commented Nov 4, 2015

jbrodriguez commented Nov 4, 2015

jbrodriguez commented Nov 4, 2015

asim commented Nov 4, 2015

jbrodriguez commented Nov 4, 2015

asim commented Nov 4, 2015

jbrodriguez commented Nov 4, 2015

asim commented Nov 4, 2015

jbrodriguez commented Nov 4, 2015