ZMQ Endpoints

The ZMQ endpoint classes are a bit difficult to understand when you first dive into them. This page attempts to demystify the basic structure of these classes and their interface.

Note that this page assumes a certain degree of proficiency with the service discovery mechanism and RTBKit in general. The article could be fleshed out a bit to explain various concepts a bit better.

Basics

• There's a set of classes for the clients (proxies) and servers (non-proxies).
• If the class has Named in its name then that means that it uses zookeeper to discover services.
  • This means that you'll be dealing with service names and classes instead of ips.
  • I'll assume that we're dealing with this class of sockets from here on out.
• To create a service-aware server socket, you use ZmqNamedEndpoint.
• To create a service-aware client socket, you use ZmqNamedProxy.
  • Note that this socket will have a zmq identity set to the name of the service.

Bus

There's this concept of a bus (not the wheeled kind) which doesn't act like a bus at all. Here's an overview of how it works.

• When a client starts up, it connects to every server on the bus. All servers on the bus will receive a notification that a new client was added.
• When a server starts up, it connects to every client on the bus. All clients on the bus will receive a notification that a new server was added.
• When sending a message both the client and the server need to specify the service name of the recipient.
• On the server side, zmq will augment any received message to identify which client sent the message. This will happen to be the service name.
• On the client side, ZmqMultipleNamedClientBusProxy will augment any received message with the service name of the server which sent the message.

Bus Servers

To connect a server to a bus, you use ZmqNamedClientBus.

• Derives from ZmqNamedEndpoint which means that zmq takes care of all the multiplexing fanciness.
• To send a message, use the sendMessage function which takes the service name of the client and the message to send. ZMQ will use the service name to route the message.
• When receiving a message, the clientMessageHandler callback will be triggered and the first argument will be the service name of the client who sent the message.
• When a new client is added or removed from the bus, the onConnection and onDisconnection callbacks are triggered with the service name of the client.
  • Connections are detected through an HELLO message and disconnections are detected through zookeeper (heartbeats didn't work out).
  • You pretty much always want to handle these events and failure to do so usually results in startup ordering problems.

Bus Clients

To connect a client to a bus, you use the infamously named ZmqMultipleNamedClientBusProxy (ZMNCBP for short... if you can call that short).

• Composed of ZmqNamedClientBusProxy which derives from ZmqNamedProxy. In this case, ZMNCBP handles all the multiplexing complexity.
• To send a message, use the sendMessage which takes the service name of the server as the first argument which will be used by the ZMNCBP to route the message to the right ZmqNamedClientBusProxy.
• When receiving a message, the messageHandler callback will be triggered with the service name of the server as the first argument.
• When a new server is added or removed from the bus, the connectHandler and disconnectHandler callbacks are triggered with the service name of the server.
  • Connections and disconnections are detected through zookeeper.
  • On the client side, the importance of these callbacks will depend on the protocol.

Conclusion

That should explain the basic interface for service-aware zmq wrapper and the rest is implementation details (it doesn't get any prettier).