Distributed PubSub with Akka Cluster
Scala
Switch branches/tags
Nothing to show
Clone or download
alexanderjarvis Merge pull request #4 from bryant1410/master
Fix broken headings in Markdown files
Latest commit 12c9ba9 Apr 17, 2017

README.md

DubSub - Distributed PubSub with Akka Cluster

Build Status

Background

PubSub, or Publish-Subscribe is a messaging pattern where the senders of messages (Publishers) have no knowledge of the receivers (Subscribers) and messages are instead categorised into 'topics' or 'channels', rather than being sent directly to specific receivers.

This loose-coupling provides for greater scalability and also provides a nice paradigm with which to design distributed systems.

Single point of bottleneck (SPOB) and more hops than a brewery

Typical implementations of the PubSub pattern denote a separate layer of network architecture, or just one server perhaps, to run as a PubSub server.

When reasoned about in a web server environment, this approach fails to make sense in terms of both performance and scalability.

Consider the following example:

Typical PubSub

Publishing a message takes two network hops to get from the second node to the first node and because all messages are routed via the Publish-Subscribe server, this can become a bottleneck.

DubSub

DubSub is designed to move the Publish-Subscribe server onto the same process, or hardware as the web server and distribute the subscriptions around a cluster of nodes automatically, using Akka Cluster.

It provides a simple Publish-Subscribe interface to work with, as a dedicated service will, yet it will route messages in a peer-to-peer fashion.

This means that you can effectively cut out the middle man and so in comparison to the typical example above, a DubSub enabled cluster (of just two nodes) will look like the following:

DubSub

How to use DubSub

There is a Play plugin available, which makes it very easy to get started if you are using this web framework.

Otherwise you can integrate DubSub into any Scala application:

Add DubSub to your project in Build.scala

libraryDependencies += "uk.co.panaxiom" % "dubsub" % "0.4-SNAPSHOT"

resolvers += Resolver.url("Alex's GitHub Repository", url("http://alexanderjarvis.github.io/snapshots/"))(Resolver.ivyStylePatterns)

On application startup, create the DubSub ActorSystem and create a DubSub actor.

import uk.co.panaxiom.dubsub._

val system = ActorSystem("DubSubSystem", config.getConfig("dubsub"))
system.actorOf(DubSub.props(), "DubSub")

Set the remote port of at least your first seed node as a runtime property (or in application.conf configuration)

-Ddubsub.akka.remote.netty.tcp.port=2551

Send Publish, Subscribe and Unsubscribe messages from any node in your cluster and watch DubSub work seamlessly.

val dubsub = system.actorFor("/user/DubSub")

dubsub ! Subscribe("topic")
dubsub ! Publish("topic", "message")
dubsub ! Unsubscribe("topic")

When subscribing to DubSub, the actor which you send the Subscribe message from is the actor that will receive any Publish message that is sent through the cluster with the same topic.

To get started on writing Akka actors, please read through the documentation at akka.io

Optional

You can specify the seed-nodes of the cluster in your application.conf:

dubsub {
  akka {
    cluster {
      seed-nodes = [
        "akka.tcp://DubSubSystem@127.0.0.1:2551",
        "akka.tcp://DubSubSystem@127.0.0.1:2552"]
    }
  }
}

How to run the tests

Just run the test command in sbt:

$ sbt
> test

DubSub uses the sbt-multi-jvm plugin to perform tests across multiple JVMs.