Switch branches/tags
Nothing to show
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
102 lines (57 sloc) 6.45 KB

Running Multicast on a Public Cloud

A little background on Multicast is worthwhile, before we start.

Multicast allows you to broadcast packets to multiple recipients via a known logical address. The address you are sending packets to is a logical destination and is not tied to any piece of hardware. Any device can then listen to that address and receive messages sent to it.

This is publish/subscribe (or pub/sub) at the IP Layer 3 level, which facilitates amongst other things auto-discovery of service. In fact this is how Apple's 'it just works' networking, Bonjour works.

But it's not all fruit cake and lollipops I'm afraid, something has to manage those logical addresses, and that would be your routers. Each of which can easily be incorrectly configured and all of which is vulnerable to Denial of Service. It's because of these concerns that multicast has had very patchy adoption.

If you're a Mac user you may have come across these problems yourself for example Bonjour doesn't work across a partitioned network, i.e. it doesn't "just work". This means that iTunes and other pieces of OS X software don't see each other. This is because something on your network is not forwarding the multicast packets correctly, usually a wireless router.

Okay, so that's the problems. Now for the advantages. If you have a working multicast environment, apps can discover each other and you get Zero-configuration networking without the need for setting up single point of failure software to discover your services through. It's a flexible solution that takes the concerns away from the application stack and places it into the network stack where it belongs.

What we need is a flat un-partitioned single tenant network.

The Cloud

As if it wasn't hard enough to run a multicast aware network in the first place, now we're putting all our apps on multi-tenant infrastructure, the cloud. In most cases cloud services simply avoid the hassle and just turn off multicast (with one notable exception Jelastic ).

So we've got this amazing automatic, configuration free, self discovery waiting to rock and roll, but we can't use it!


After the advent of Docker many of us are starting to look at things in a different way. Our apps are now the same as machines thanks to Docker. This means our applications can act like they once did, as the sole piece of software running on a machine. Which has the interesting side effect of making older and low level technologies directly relevant to application development and operations.

Forget your UDDI, you can go back to using DNS (optionally with SRV records) instead. Forget application servers, just load balance your containers. It is as if we have turned back the clock 30 years, in a good way.

And in this new world order of Docker, distributed service discovery can now be done simply with multicast.


So what we really need to make all this work is a way to overlay the existing network in such a way as to get a single tenant unpartioned, flat network. This is where Weave now comes in. Weave provides an overlay network which links Docker containers together and allows any Layer 2 protocol over it, including multicast!

Let's Get Down to it Boppers

To get the most out of learning about multicast and weave you should really get a Digital Ocean Account ($10 off with that link) - that way you can run a single script to get the demo running. The full tutorial can be found on Github including a single script deployment. But for this blog post we're going to go through the steps one at a time.

Show me the code!

You've waited patiently so here we go, let's install weave on each of our machines:

sudo wget -O /usr/local/bin/weave \
sudo chmod a+x /usr/local/bin/weave

Next install ethtool and conntrack on them, if you're on a Debian system such as Ubuntu, this will work:

apt-get -y install ethtool conntrack

Now we need to start up Weave using a private network address. On the first host enter:

weave launch ""

This says that the host we're on now will have an ip address of assigned to it on the Weave network and we're interested in other ip addresses in the range

On the second host set the variable seed_host to be equal to the external IP address of the first host. Then run

weave launch "" ${seed_host}

We have set up two hosts that can see each other so let's check what's happening, on either host enter:

weave status

If all is okay we can now start our Docker container on the first host:

CONTAINER=$(weave run -t -i neilellis/weave-multicast-tutorial / Mal) docker attach $CONTAINER

And the second host:

CONTAINER=$(weave run -t -i neilellis/weave-multicast-tutorial / Jayne)  docker attach $CONTAINER

That's it, you should now have a working chat system using multicast to connect your machines across a public cloud network, all thanks to Weave!

If you're lazy (like me) and just want to see it working, then go and get your Digital Ocean Account, set up Tugboat on your machine then just:

git clone
cd weave_multicast_tutorial

When you're finished remove the droplets with:


Obviously creating droplets will incur charges.

All the best

Neil Ellis

The example C code used in the Docker example comes from