Multicast overlays

With gossiperl, it is possible to start an overlay over multicast. To do so, simply run this command:

curl -i -k -u $REST_USER:$REST_PASS \
  -X POST \
  -H 'Contetnt-Type: application/json; charset=utf-8' -d '{
  "multicast": {
    "ip": "224.0.0.251",
    "ttl": 4,
    "local_iface_address": "0.0.0.0"
  },
  "port": 6666,
  "symmetric_key": "v3JElaRswYgxOt4b"
}' https://192.168.50.100:8080/overlays/gossiper_overlay_remote

Supported multicast options are:

• ip: IP address of the multicast group, required if multicast option is given
• ttl: TTL for outgoing multicast datagrams in order to control the scope of the multicasts, not required, default is 4
• local_iface_address: multicast interface address, not required, default is 0.0.0.0
• local_port: port on which gossiperl overlay binds for connections on 127.0.0.1, default is <overlayPort>+1

Ignored configuration settings

If a multicast overlay is requested, the following settings are ignored:

• <overlayConfig>.ip: <multicast>.ip is used
• <overlayConfig>.rack_name: can be given but not used for anything in a multicast overlay
• <overlayConfig>.racks: can be given but not used, also not validated in a multicast overlay

Local clients

A multicast overlay will attempting binding itself on two different ports:

• <multicast>.ip:<overlayConfig>.port
• 127.0.0.1:<multicast>.local_port

The reason for this is the following: the whole concept of gossiperl is to separate an overlay and local members. When an overlay is running in multicast, the local members, if they were to send data via the multicast address, this data would be broadcast to every overlay member. To solve this problem, an overlay also binds itself to 127.0.0.1:<multicast>.local_port. The clients still connect to an overlay member via localhost, the data is then transferred between local and multicast connections as needed.

If the overlay is configured with an IPv6 address, the local connection will be bound to ::1 instead of 127.0.0.1.

All gossiperl settings like secret, quarantine_after, and so on, are used for both connections. Reconfiguring an overlay affects both connections (but multicast settings apply to multicast connection only).

Setting up multicast on Ubuntu

On Ubuntu it is as simple as:

route add -net 224.0.0.0 netmask 224.0.0.0 eth1

Obviously one can choose a network address to use, this above example uses 224.0.0.0.

To check if multicast is available:

ifconfig eth(X)
# and look for: MULTICAST

To enable multicast:

ifconfig eth(X) multicast

To disable:

ifconfig eth(X) -multicast

Setting up multicast on OS X

First, check if multicast is enabled:

ifconfig en0

en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        ether 28:cf:e9:13:8d:8d
        inet6 fe80::2acf:e9ff:fe13:8d8d%en0 prefixlen 64 scopeid 0x4
        inet 10.128.30.26 netmask 0xffffff00 broadcast 10.128.30.255
        nd6 options=1<PERFORMNUD>
        media: autoselect
        status: active

If flags contains MULTICAST, multicast is available. Time to check if multicast routing is available:

netstat -nr

If the Internet table does not contain 224.0.0.0 – 239.255.255.255 route, a multicast address has to be added to the routes:

sudo route -nv add -net 224.0.0.251 -interface en0
netstat -nr | grep 224.0.0.

224.0.0.251/32     1:0:5e:0:0:fb      UmLS            0        2     en0

To delete the route:

sudo route -v delete -inet 224.0.0.251