New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

node-discover for nodes which are not in the same LAN #17

Closed
PavelPolyakov opened this Issue Oct 23, 2015 · 9 comments

Comments

Projects
None yet
3 participants
@PavelPolyakov

Hi,

Sorry for asking the question which is probably obvious, but - as I understand this discovery thing is working perfect, when all the nodes are located within one LAN network.

What if I want to run the nodes in the different LANs, but still want them to find each other. Is there anything which is possible to configure using node-discover for such case?

For example, using ElasticSearch, I am able to state the IPs (where I expect the other nodes to be run) in the configuration, so cluster will gather itself:

discovery.zen.ping.unicast.hosts: ["xxx","yyy","zzz"]

Regards,

@solgarius

This comment has been minimized.

Show comment
Hide comment
@solgarius

solgarius Oct 23, 2015

Hi,
I'm going to preface this comment with: I could be completely wrong about this, because its just what I have read/understand as opposed to something I have tested myself.

As far as I know node-discover will only work on the same network (sub-net?) where a broadcast/unicast packet can be picked up.

Also I was under the impression that AWS doesn't support broadcast/unicast packets, so I don't believe the node-discover style of module discovery will work?

More than happy to be proved wrong on this.

Hi,
I'm going to preface this comment with: I could be completely wrong about this, because its just what I have read/understand as opposed to something I have tested myself.

As far as I know node-discover will only work on the same network (sub-net?) where a broadcast/unicast packet can be picked up.

Also I was under the impression that AWS doesn't support broadcast/unicast packets, so I don't believe the node-discover style of module discovery will work?

More than happy to be proved wrong on this.

@PavelPolyakov

This comment has been minimized.

Show comment
Hide comment
@PavelPolyakov

PavelPolyakov Oct 23, 2015

@solgarius
Yes, that is most probably true, but maybe there is some solution or library which allows us to build clusters with kind of auto discovery for the nodes which are located in different networks.

@solgarius
Yes, that is most probably true, but maybe there is some solution or library which allows us to build clusters with kind of auto discovery for the nodes which are located in different networks.

@wankdanker

This comment has been minimized.

Show comment
Hide comment
@wankdanker

wankdanker Oct 23, 2015

Owner

@PavelPolyakov, in its current incarnation, it is limited to the broadcast domain or however multicast works which may go beyond the lan depending on your router.

I have considered the idea of being able to specify an array of addresses to which messages should be sent. It actually would be very easy to implement. I'll give it a shot and you can try it out if you like.

Owner

wankdanker commented Oct 23, 2015

@PavelPolyakov, in its current incarnation, it is limited to the broadcast domain or however multicast works which may go beyond the lan depending on your router.

I have considered the idea of being able to specify an array of addresses to which messages should be sent. It actually would be very easy to implement. I'll give it a shot and you can try it out if you like.

@PavelPolyakov

This comment has been minimized.

Show comment
Hide comment
@PavelPolyakov

PavelPolyakov Oct 23, 2015

@wankdanker
Thanks, sounds promising!

@wankdanker
Thanks, sounds promising!

@wankdanker

This comment has been minimized.

Show comment
Hide comment
@wankdanker

wankdanker Oct 23, 2015

Owner

@PavelPolyakov, v0.4.0 is on npm and has a unicast option. It takes a comma separated string or array of addresses.

This change would have been super quick, but I noticed some odd behavior when testing locally. I had to set up a virtual network interface so that my local machine had two IP addresses. If you test with examples/basic-argv.js you can do something like:

Instance 1

node examples/basic-argv --port=26364 --address=192.168.1.100 --unicast=192.168.1.100,192.168.1.101

Instance 2

node examples/basic-argv --port=26364 --address=192.168.1.101 --unicast=192.168.1.100,192.168.1.101
Owner

wankdanker commented Oct 23, 2015

@PavelPolyakov, v0.4.0 is on npm and has a unicast option. It takes a comma separated string or array of addresses.

This change would have been super quick, but I noticed some odd behavior when testing locally. I had to set up a virtual network interface so that my local machine had two IP addresses. If you test with examples/basic-argv.js you can do something like:

Instance 1

node examples/basic-argv --port=26364 --address=192.168.1.100 --unicast=192.168.1.100,192.168.1.101

Instance 2

node examples/basic-argv --port=26364 --address=192.168.1.101 --unicast=192.168.1.100,192.168.1.101
@PavelPolyakov

This comment has been minimized.

Show comment
Hide comment
@PavelPolyakov

PavelPolyakov Oct 23, 2015

We have just tested it, and have the next issue (probably it's expected behaviour, but not for me).

I can only use the IP address which was provided for me by DHCP server, not the one which the server is available from the internet.

pavel@iii:~/node-discover$ ifconfig
eth0      Link encap:Ethernet  HWaddr xxxx
          inet addr:10.95.166.12  Bcast:10.95.166.63  Mask:255.255.255.192
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1782617842 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1417219418 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:411603809362 (411.6 GB)  TX bytes:243805862358 (243.8 GB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:569282324 errors:0 dropped:0 overruns:0 frame:0
          TX packets:569282324 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:33895867796 (33.8 GB)  TX bytes:33895867796 (33.8 GB)

(in the example above, I can use the "inet addr").

If I am trying to use the internet available address, I get the next error:
image

I have tried the same being on local machine.
We have successfully created the cluster with my friend, while both were using the next IP:
image

Any thoughts is this right or wrong?

We have just tested it, and have the next issue (probably it's expected behaviour, but not for me).

I can only use the IP address which was provided for me by DHCP server, not the one which the server is available from the internet.

pavel@iii:~/node-discover$ ifconfig
eth0      Link encap:Ethernet  HWaddr xxxx
          inet addr:10.95.166.12  Bcast:10.95.166.63  Mask:255.255.255.192
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1782617842 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1417219418 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:411603809362 (411.6 GB)  TX bytes:243805862358 (243.8 GB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:569282324 errors:0 dropped:0 overruns:0 frame:0
          TX packets:569282324 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:33895867796 (33.8 GB)  TX bytes:33895867796 (33.8 GB)

(in the example above, I can use the "inet addr").

If I am trying to use the internet available address, I get the next error:
image

I have tried the same being on local machine.
We have successfully created the cluster with my friend, while both were using the next IP:
image

Any thoughts is this right or wrong?

@wankdanker

This comment has been minimized.

Show comment
Hide comment
@wankdanker

wankdanker Oct 23, 2015

Owner

The --address is just an indicator to tell the process which local interface to bind to. So, in your case it should be:

node examples/basic-argv --port=26364 --address=10.95.166.12 --unicast=54...

And you will need to make sure that the remote (54...) machine is allowing incoming UDP messages on port 26364. Likewise if you are running this same example on the remote (54...) pointing back at your local machine (via your public IP address), you would need to make sure your router/firewall is allowing incoming UDP on 26364 and forwarding that traffic to 10.95.166.12.

I'm not entirely sure of your setup, but let's pretend based on what I'm picturing with fake addresses

Local Setup

Machine IP: 10.1.1.1
Router/Public IP: 54.1.1.1
Router forwards UDP on port 55555 to port 55555 on 10.1.1.1
Command:

node examples/basic-argv --port=55555 --address=10.1.1.1 --unicast=54.1.1.1,54.2.2.2

Remote Setup

Machine IP: 10.100.100.100
Router/Public IP: 54.2.2.2
Router forwards UDP on port 55555 to port 55555 on 10.100.100.100
Command:

node examples/basic-argv --port=55555 --address=10.100.100.100 --unicast=54.1.1.1,54.2.2.2

This scenario assumes both machines are behind a router/firewall doing NAT. Not sure how close this is to your setup.

Owner

wankdanker commented Oct 23, 2015

The --address is just an indicator to tell the process which local interface to bind to. So, in your case it should be:

node examples/basic-argv --port=26364 --address=10.95.166.12 --unicast=54...

And you will need to make sure that the remote (54...) machine is allowing incoming UDP messages on port 26364. Likewise if you are running this same example on the remote (54...) pointing back at your local machine (via your public IP address), you would need to make sure your router/firewall is allowing incoming UDP on 26364 and forwarding that traffic to 10.95.166.12.

I'm not entirely sure of your setup, but let's pretend based on what I'm picturing with fake addresses

Local Setup

Machine IP: 10.1.1.1
Router/Public IP: 54.1.1.1
Router forwards UDP on port 55555 to port 55555 on 10.1.1.1
Command:

node examples/basic-argv --port=55555 --address=10.1.1.1 --unicast=54.1.1.1,54.2.2.2

Remote Setup

Machine IP: 10.100.100.100
Router/Public IP: 54.2.2.2
Router forwards UDP on port 55555 to port 55555 on 10.100.100.100
Command:

node examples/basic-argv --port=55555 --address=10.100.100.100 --unicast=54.1.1.1,54.2.2.2

This scenario assumes both machines are behind a router/firewall doing NAT. Not sure how close this is to your setup.

@PavelPolyakov

This comment has been minimized.

Show comment
Hide comment
@PavelPolyakov

PavelPolyakov Oct 23, 2015

Correct, this concept had worked for me, and I was connected to the remote:

Local:

node examples/basic-argv --port=26364 --address=10.91.16.14 --unicast=10.91.16.14,54.xx.xx.xx

Remote:

node examples/basic-argv --port=26364 --address=10.95.166.12 --unicast=54.xx.xx.xx

Thanks for the library and for the quick support, all of this is really interesting!

Correct, this concept had worked for me, and I was connected to the remote:

Local:

node examples/basic-argv --port=26364 --address=10.91.16.14 --unicast=10.91.16.14,54.xx.xx.xx

Remote:

node examples/basic-argv --port=26364 --address=10.95.166.12 --unicast=54.xx.xx.xx

Thanks for the library and for the quick support, all of this is really interesting!

@wankdanker

This comment has been minimized.

Show comment
Hide comment
@wankdanker

wankdanker Oct 23, 2015

Owner

Sure thing. :)

I've pondered the idea of having a discover proxy or something that could pass along discovery messages across network boundaries. Haven't thought too much about it but maybe it'll happen at some point.

Have fun!

Owner

wankdanker commented Oct 23, 2015

Sure thing. :)

I've pondered the idea of having a discover proxy or something that could pass along discovery messages across network boundaries. Haven't thought too much about it but maybe it'll happen at some point.

Have fun!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment