👍

Zero-config external clients!

I'd be interested in doing this for the Perl API too

I updated the description with more info as to the format of the request/response.

Pushed to master, would love to get feedback on this by "other" clients :)

@kimchy: Sounds great!, we're in a hot discussion over at karmi/retire#162 :)

Hiya

Tried this out, and ES is responding, but without the HTTP address:

'{"response":{"cluster_name":"iAnnounce-demo","transport_address":"inet[/192.168.5.10:9300]","attributes":{}}}

To be clear, what I'm doing in Perl is:

use IO::Socket::Multicast; 
my $s=IO::Socket::Multicast->new(
    LocalPort   => 54328, 
    LocalAddr   => '224.2.2.4',
    ReuseAddr   => 1
); 
$s->mcast_add('224.2.2.4');
$s->mcast_send('{"request":{"cluster_name":"iAnnounce-demo"}}','224.2.2.4:54328'); 

$s->recv($d,1024);
# '{"request":{"cluster_name":"iAnnounce-demo"}}'

$s->recv($d,1024);
# '{"response":{"cluster_name":"iAnnounce-demo","transport_address":"inet[/192.168.5.10:9300]","attributes":{}}}'

using this module: https://metacpan.org/module/IO::Socket::Multicast

@kimchy This is awesome. I am going to try it out shortly. Is it possible to have the cluster multicast the same json format when a node comes online or goes offline? This will allow the client to just listen for updates to know active clients rather than pinging all the time.

@clintongormley it should be fixed now

@johnwilliams thats a bit different, since the multicast will not be a message of all nodes in the cluster (possibly sent by the master) on changes to the topology, whereby now, it simply reacts to a direct request and sends back that node info.

@kimchy I'm trying to think of how to best incorporate multicast discovery in the Perl API.

Currently, we connect to a cluster using a configured server address. We extract the list of live nodes and store them for (a) round robin'ing and (b) to re-sniff if any node goes down.

We can only use multicast if the user provides a cluster name. I think that if they provide a cluster name AND one or more server addresses, then we should still use the server address in favour of multicast (but possibly confirm that the cluster name is the same?).

Question comes when (a) we've sniffed the cluster using multicast and (b) when one of the sniffed nodes disappears. Do we:
(1) use the current mechanism of doing a /_nodes request against each of the previously sniffed nodes, or
(2) use multicast to re-sniff instead.

My instinct would be to use multicast, as it kinda works in parallel (ie any server could respond), even if our code is sync, while the /_nodes requests would have to be done serially. So in the case that one node is hanging (eg a broken switch or bad firewalling), with multicast we would avoid the delay of waiting for that node to timeout.

If using multicast is better, then even if the user specifies a static server list initially, should we extract and store the cluster name and use multicast for future sniffing? It may be that multicast is disabled on the network, but we won't know that until we try it out. (of course, could add an option to disable multicast in the client)

Another question is: we're likely to get multiple multicast responses (1 from each server). Do we just accept the first one, or do we wait for any responses received within a certain timeout? In the case of split brain, we may get different responses, and want to connect to the cluster which has more nodes.

We can only use multicast if the user provides a cluster name. I think that if they provide a cluster name AND one or more server addresses, then we should still use the server address in favour of multicast (but possibly confirm that the cluster name is the same?).

Yes, make sense. I think if cluster name is provided, and a list of servers is also provided, it makes sense to verify that the cluster name is the same.

Regarding the re-sniffing if a node fails. I think that as long as you have live nodes, use the /nodes API (and you only need to execute it once, no need to do that on each node). Once you no longer have any more nodes, to connect to, use multicast.

i can use this hot feature in my client,great~