Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

Various typeface fixes

  • Loading branch information...
commit a5efbcadbd40125548dc7c5bacf3d9226b3adbe1 1 parent 837932b
@hintjens hintjens authored
Showing with 78 additions and 80 deletions.
  1. +7 −7 chapter2.txt
  2. +69 −71 chapter3.txt
  3. +2 −2 chapter8.txt
View
14 chapter2.txt
@@ -202,7 +202,7 @@ One of the things we aim to provide you with in this book are a set of such high
+++ Working with Messages
-The libzmq core library has in fact two APIs to send and receive messages. The {{zmq_send[3]}} and {{zmq_recv[3]}} methods that we've already seen and used are simple one-liners. We will use these often, but {{zmq_recv[3]}} is bad at dealing with arbitrary message sizes: it truncates messages to whatever buffer size you provide. So there's a second API that works with zmq_msg_t structures, with a richer but more difficult API:
+The {{libzmq}} core library has in fact two APIs to send and receive messages. The {{zmq_send[3]}} and {{zmq_recv[3]}} methods that we've already seen and used are simple one-liners. We will use these often, but {{zmq_recv[3]}} is bad at dealing with arbitrary message sizes: it truncates messages to whatever buffer size you provide. So there's a second API that works with zmq_msg_t structures, with a richer but more difficult API:
* Initialise a message: {{zmq_msg_init[3]}}, {{zmq_msg_init_size[3]}}, {{zmq_msg_init_data[3]}}.
* Sending and receiving a message: {{zmq_msg_send[3]}}, {{zmq_msg_recv[3]}}.
@@ -716,7 +716,7 @@ Here is the modified sink application. When it's finished collecting results, it
++ Handling Interrupt Signals
-Realistic applications need to shut down cleanly when interrupted with Ctrl-C or another signal such as SIGTERM. By default, these simply kill the process, meaning messages won't be flushed, files won't be closed cleanly, and so on.
+Realistic applications need to shut down cleanly when interrupted with Ctrl-C or another signal such as {{SIGTERM}}. By default, these simply kill the process, meaning messages won't be flushed, files won't be closed cleanly, and so on.
Here is how we handle a signal in various languages:
@@ -743,7 +743,7 @@ while (!s_interrupted) {
zmq_close (client);
[[/code]]
-If you call {{s_catch_signals()}} and don't test for interrupts, then your application will become immune to Ctrl-C and SIGTERM, which may be useful, but is usually not.
+If you call {{s_catch_signals()}} and don't test for interrupts, then your application will become immune to Ctrl-C and {{SIGTERM}}, which may be useful, but is usually not.
++ Detecting Memory Leaks
@@ -1031,7 +1031,7 @@ zmq_msg_init_data (&message, buffer, 1000, my_free, NULL);
zmq_msg_send (socket, &message, 0);
[[/code]]
-Note that you don't call {{zmq_msg_close[3]}} after sending a message--libzmq will do this automatically when it's actually done sending the message.
+Note that you don't call {{zmq_msg_close[3]}} after sending a message--{{libzmq}} will do this automatically when it's actually done sending the message.
There is no way to do zero-copy on receive: 0MQ delivers you a buffer that you can store as long as you wish, but it will not write data directly into application buffers.
@@ -1070,6 +1070,8 @@ When you run the two programs, the subscriber should show you this:
...
[[/code]]
+This example shows that the subscription filter rejects or accepts the entire multipart message (key plus data). You won't get part of a multipart message, ever. If you subscribe to multiple publishers and you want to know their address so that you can send them data via another socket (and this is a typical use case), create a three-part message[figure].
+
[[code type="textdiagram" title="Pub-Sub Envelope with Sender Address"]]
#---------#
Frame 1 | Key | Subscription key
@@ -1080,8 +1082,6 @@ Frame 3 | Data | Actual message body
#---------#
[[/code]]
-This example shows that the subscription filter rejects or accepts the entire multipart message (key plus data). You won't get part of a multipart message, ever. If you subscribe to multiple publishers and you want to know their address so that you can send them data via another socket (and this is a typical use case), create a three-part message[figure].
-
++ High-Water Marks
When you can send messages rapidly from process to process, you soon discover that memory is a precious resource, and one that can be trivially filled up. A few seconds of delay somewhere in a process can turn into a backlog that blows up a server unless you understand the problem and take precautions.
@@ -1100,7 +1100,7 @@ In 0MQ v2.x, the HWM was infinite by default. This was easy but also typically f
When your socket reaches its HWM, it will either block or drop data depending on the socket type. PUB and ROUTER sockets will drop data if they reach their HWM, while other socket types will block. Over the {{inproc}} transport, the sender and receiver share the same buffers, so the real HWM is the sum of the HWM set by both sides.
-Lastly, the HWMs are not exact; while you may get //up to// 1,000 messages by default, the real buffer size may be much lower (as little as half), due to the way libzmq implements its queues.
+Lastly, the HWMs are not exact; while you may get //up to// 1,000 messages by default, the real buffer size may be much lower (as little as half), due to the way {{libzmq}} implements its queues.
++ Missing Message Problem Solver
View
140 chapter3.txt
@@ -83,7 +83,7 @@ The {{zmq_socket[3]}} man page describes it thus:
> When receiving messages a ZMQ_ROUTER socket shall prepend a message part containing the identity of the originating peer to the message before passing it to the application. Messages received are fair-queued from among all connected peers. When sending messages a ZMQ_ROUTER socket shall remove the first part of the message and use it to determine the identity of the peer the message shall be routed to.
-As a historical note, 0MQ v2.2 and earlier use UUIDs as identities, and 0MQ v3.0 and later use short integers. There's some impact on network performance, but only when you use multiple proxy hops, which is rare. Mostly the change was to simplify building libzmq by removing the dependency on a UUID library.
+As a historical note, 0MQ v2.2 and earlier use UUIDs as identities, and 0MQ v3.0 and later use short integers. There's some impact on network performance, but only when you use multiple proxy hops, which is rare. Mostly the change was to simplify building {{libzmq}} by removing the dependency on a UUID library.
Identies are a difficult concept to understand, but it's essential if you want to become a 0MQ expert. The ROUTER socket //invents// a random identity for each connection with which it works. If there are three REQ sockets connected to a ROUTER socket, it will invent three random identities, one for each REQ socket.
@@ -273,7 +273,7 @@ Here is what the program prints:
ROUTER sockets do have a somewhat brutal way of dealing with messages they can't send anywhere: they drop them silently. It's an attitude that makes sense in working code, but it makes debugging hard. The "send identity as first frame" approach is tricky enough that we often get this wrong when we're learning, and the ROUTER's stony silence when we mess up isn't very constructive.
-Since 0MQ v3.2 there's a socket option you can set to catch this error: {{ZMQ_ROUTER_MANDATORY}}. Set that on the ROUTER socket and then when you provide an unroutable identity on a send call, the socket will signal an EHOSTUNREACH error.
+Since 0MQ v3.2 there's a socket option you can set to catch this error: {{ZMQ_ROUTER_MANDATORY}}. Set that on the ROUTER socket and then when you provide an unroutable identity on a send call, the socket will signal an {{EHOSTUNREACH}} error.
++ The Load Balancing Pattern
@@ -317,6 +317,8 @@ Completed: 25 tasks
Completed: 19 tasks
[[/code]]
+To talk to the workers in this example, we have to create a REQ-friendly envelope consisting of an identity plus an empty envelope delimiter frame[figure].
+
[[code type="textdiagram" title="Routing Envelope for REQ"]]
#---+-------#
Frame 1 | n | ... | Identity of connection
@@ -327,8 +329,6 @@ Frame 3 | n | ... | Data frame
#---+--------#
[[/code]]
-To talk to the workers in this example, we have to create a REQ-friendly envelope consisting of an identity plus an empty envelope delimiter frame[figure].
-
+++ ROUTER Broker and DEALER Workers
Anywhere you can use REQ, you can use DEALER. There are two specific differences:
@@ -351,6 +351,8 @@ If we never need to pass the message along to a REP socket, we can simply drop t
+++ A Load Balancing Message Broker
+The previous example is half-complete. It can manage a set of workers with dummy requests and replies, but it has no way to talk to clients. If we add a second //frontend// ROUTER socket that accepts client requests, and turn our example into a proxy that can switch messages from frontend to backend, we get a useful and reusable tiny load balancing message broker[figure].
+
[[code type="textdiagram" title="Load Balancing Broker"]]
#--------# #--------# #--------#
| Client | | Client | | Client |
@@ -377,10 +379,6 @@ If we never need to pass the message along to a REP socket, we can simply drop t
#--------# #--------# #--------#
[[/code]]
-The previous example is half-complete. It can manage a set of workers with dummy requests and replies, but it has no way to talk to clients.
-
-If we add a second //frontend// ROUTER socket that accepts client requests, and turn our example into a proxy that can switch messages from frontend to backend, we get a useful and reusable tiny load balancing message broker[figure].
-
This broker does the following:
* Accepts connections from a set of clients.
@@ -397,13 +395,15 @@ The broker code is fairly long, but worth understanding:
The difficult part of this program is (a) the envelopes that each socket reads and writes, and (b) the load balancing algorithm. We'll take these in turn, starting with the message envelope formats.
+Let's walk through a full request-reply chain from client to worker and back. In this code we set the identity of client and worker sockets to make it easier to trace the message frames. In reality, we'd allow the ROUTER sockets to invent identities for connections. Let's assume the client's identity is "CLIENT" and the worker's identity is "WORKER". The client application sends a single frame containing "Hello"[figure].
+
[[code type="textdiagram" title="Message that Client Sends"]]
#---+-------#
Frame 1 | 5 | Hello | Data frame
#---+-------#
[[/code]]
-Let's walk through a full request-reply chain from client to worker and back. In this code we set the identity of client and worker sockets to make it easier to trace the message frames. In reality, we'd allow the ROUTER sockets to invent identities for connections. Let's assume the client's identity is "CLIENT" and the worker's identity is "WORKER". The client application sends a single frame containing "Hello"[figure].
+Because the REQ socket adds its empty delimiter frame and the ROUTER socket adds its connection identity, the proxy reads off the frontend ROUTER socket the client address, empty delimiter frame, and the data part[figure].
[[code type="textdiagram" title="Message Coming in on Frontend"]]
#---+--------#
@@ -415,7 +415,7 @@ Frame 3 | 5 | Hello | Data frame
#---+-------#
[[/code]]
-Because the REQ socket adds its empty delimiter frame and the ROUTER socket adds its connection identity, the proxy reads off the frontend ROUTER socket the client address, empty delimiter frame, and the data part[figure].
+The broker sends this to the worker, prefixed by the address of the chosen worker, plus an additional empty part to keep the REQ at the other end happy[figure].
[[code type="textdiagram" title="Message Sent to Backend"]]
#---+--------#
@@ -431,12 +431,8 @@ Frame 5 | 5 | Hello | Data frame
#---+-------#
[[/code]]
-The broker sends this to the worker, prefixed by the address of the chosen worker, plus an additional empty part to keep the REQ at the other end happy[figure].
-
This complex envelope stack gets chewed up first by the backend ROUTER socket, which removes the first frame. Then the REQ socket in the worker removes the empty part, and provides the rest to the worker application[figure].
-The worker has to save the envelope (which is all the parts up to and including the empty message frame) and then it can do what's needed with the data part. Note that a REP socket would do this automatically, but we're using the REQ-ROUTER pattern so that we can get proper load balancing.
-
[[code type="textdiagram" title="Message Delivered to Worker"]]
#---+--------#
Frame 1 | 6 | CLIENT | Identity of client
@@ -447,6 +443,8 @@ Frame 3 | 5 | Hello | Data frame
#---+-------#
[[/code]]
+The worker has to save the envelope (which is all the parts up to and including the empty message frame) and then it can do what's needed with the data part. Note that a REP socket would do this automatically, but we're using the REQ-ROUTER pattern so that we can get proper load balancing.
+
On the return path, the messages are the same as when they come in, i.e., the backend socket gives the broker a message in five parts, and the broker sends the frontend socket a message in three parts, and the client gets a message in one part.
Now let's look at the load balancing algorithm. It requires that both clients and workers use REQ sockets, and that workers correctly store and replay the envelope on messages they get. The algorithm is:
@@ -487,7 +485,7 @@ while (true) {
}
[[/code]]
-That code isn't even reusable because it can only handle one reply address in the envelope, and it already does some wrapping around the 0MQ API. If we used the libzmq simple message API this is what we'd have to write:
+That code isn't even reusable because it can only handle one reply address in the envelope, and it already does some wrapping around the 0MQ API. If we used the {{libzmq}} simple message API this is what we'd have to write:
[[code type="fragment" name="lowreader"]]
while (true) {
@@ -563,7 +561,7 @@ Here is the load balancing broker rewritten to use a higher-level API (CZMQ for
[[code type="example" title="Load balancing broker using high-level API" name="lbbroker2"]]
[[/code]]
-One thing CZMQ provides is clean interrupt handling. This means that Ctrl-C will cause any blocking 0MQ call to exit with a return code -1 and errno set to EINTR. The high-level recv methods will return NULL in such cases. So, you can cleanly exit a loop like this:
+One thing CZMQ provides is clean interrupt handling. This means that Ctrl-C will cause any blocking 0MQ call to exit with a return code -1 and errno set to {{EINTR}}. The high-level recv methods will return NULL in such cases. So, you can cleanly exit a loop like this:
[[code type="fragment" name="interrupt"]]
while (true) {
@@ -613,6 +611,8 @@ Getting applications to properly shut down when you send them Ctrl-C can be tric
++ The Asynchronous Client/Server Pattern
+In the ROUTER to DEALER example, we saw a 1-to-N use case where one server talks asynchronously to multiple workers. We can turn this upside down to get a very useful N-to-1 architecture where various clients talk to a single server, and do this asynchronously[figure].
+
[[code type="textdiagram" title="Asynchronous Client/Server"]]
#----------# #----------#
| Client | | Client |
@@ -631,8 +631,6 @@ Getting applications to properly shut down when you send them Ctrl-C can be tric
#-------------#
[[/code]]
-In the ROUTER to DEALER example, we saw a 1-to-N use case where one server talks asynchronously to multiple workers. We can turn this upside down to get a very useful N-to-1 architecture where various clients talk to a single server, and do this asynchronously[figure].
-
Here's how it works:
* Clients connect to the server and send requests.
@@ -647,6 +645,12 @@ Here's code that shows how this works:
The example runs in one process, with multiple threads simulating a real multiprocess architecture. When you run the example, you'll see three clients (each with a random ID), printing out the replies they get from the server. Look carefully and you'll see each client task gets 0 or more replies per request.
+Some comments on this code:
+
+* The clients send a request once per second, and get zero or more replies back. To make this work using {{zmq_poll[3]}}, we can't simply poll with a 1-second timeout, or we'd end up sending a new request only one second //after we received the last reply//. So we poll at a high frequency (100 times at 1/100th of a second per poll), which is approximately accurate.
+
+* The server uses a pool of worker threads, each processing one request synchronously. It connects these to its frontend socket using an internal queue[figure]. It connects the frontend and backend sockets using a {{zmq_proxy[3]}} call.
+
[[code type="textdiagram" title="Detail of Asynchronous Server"]]
#---------# #---------# #---------#
| Client | | Client | | Client |
@@ -681,12 +685,6 @@ The example runs in one process, with multiple threads simulating a real multipr
'-------------------------------------------'
[[/code]]
-Some comments on this code:
-
-* The clients send a request once per second, and get zero or more replies back. To make this work using {{zmq_poll[3]}}, we can't simply poll with a 1-second timeout, or we'd end up sending a new request only one second //after we received the last reply//. So we poll at a high frequency (100 times at 1/100th of a second per poll), which is approximately accurate.
-
-* The server uses a pool of worker threads, each processing one request synchronously. It connects these to its frontend socket using an internal queue[figure]. It connects the frontend and backend sockets using a {{zmq_proxy[3]}} call.
-
Note that we're doing DEALER to ROUTER dialog between client and server, but internally between the server main thread and workers, we're doing DEALER to DEALER. If the workers were strictly synchronous, we'd use REP. However, because we want to send multiple replies, we need an async socket. We do //not// want to route replies, they always go to the single server thread that sent us the request.
Let's think about the routing envelope. The client sends a message consisting of a single frame. The server thread receives a two-frame message (original message prefixed by client identity). We send these two frames on to the worker, which treats it as a normal reply envelope, returns that to us as a two frame message. We then use the first frame as an identity to route the second frame back to the client as a reply.
@@ -743,6 +741,12 @@ So we do a little calculation and see that this will work nicely over plain TCP.
It's a straightforward problem that requires no exotic hardware or protocols, just some clever routing algorithms and careful design. We start by designing one cluster (one data center) and then we figure out how to connect clusters together.
++++ Architecture of a Single Cluster
+
+Workers and clients are synchronous. We want to use the load balancing pattern to route tasks to workers. Workers are all identical; our facility has no notion of different services. Workers are anonymous; clients never address them directly. We make no attempt here to provide guaranteed delivery, retry, and so on.
+
+For reasons we already examined, clients and workers won't speak to each other directly. It makes it impossible to add or remove nodes dynamically. So our basic model consists of the request-reply message broker we saw earlier[figure].
+
[[code type="textdiagram" title="Cluster Architecture"]]
#--------# #--------# #--------#
| Client | | Client | | Client |
@@ -770,12 +774,6 @@ It's a straightforward problem that requires no exotic hardware or protocols, ju
#--------# #--------# #--------#
[[/code]]
-+++ Architecture of a Single Cluster
-
-Workers and clients are synchronous. We want to use the load balancing pattern to route tasks to workers. Workers are all identical; our facility has no notion of different services. Workers are anonymous; clients never address them directly. We make no attempt here to provide guaranteed delivery, retry, and so on.
-
-For reasons we already examined, clients and workers won't speak to each other directly. It makes it impossible to add or remove nodes dynamically. So our basic model consists of the request-reply message broker we saw earlier[figure].
-
+++ Scaling to Multiple Clusters
Now we scale this out to more than one cluster. Each cluster has a set of clients and workers, and a broker that joins these together[figure].
@@ -808,10 +806,6 @@ The question is: how do we get the clients of each cluster talking to the worker
Let's explore Idea #1. In this model, we have workers connecting to both brokers and accepting jobs from either one[figure].
-It looks feasible. However, it doesn't provide what we wanted, which was that clients get local workers if possible and remote workers only if it's better than waiting. Also workers will signal "ready" to both brokers and can get two jobs at once, while other workers remain idle. It seems this design fails because again we're putting routing logic at the edges.
-
-So, idea #2 then. We interconnect the brokers and don't touch the clients or workers, which are REQs like we're used to[figure].
-
[[code type="textdiagram" title="Idea 1: Cross-connected Workers"]]
Cluster 1 : Cluster 2
:
@@ -833,15 +827,9 @@ So, idea #2 then. We interconnect the brokers and don't touch the clients or wor
#--------# #--------# #--------# :
[[/code]]
-This design is appealing because the problem is solved in one place, invisible to the rest of the world. Basically, brokers open secret channels to each other and whisper, like camel traders, "Hey, I've got some spare capacity. If you have too many clients, give me a shout and we'll deal".
-
-In effect it is just a more sophisticated routing algorithm: brokers become subcontractors for each other. There are other things to like about this design, even before we play with real code:
-
-* It treats the common case (clients and workers on the same cluster) as default and does extra work for the exceptional case (shuffling jobs between clusters).
-
-* It lets us use different message flows for the different types of work. That means we can handle them differently, e.g., using different types of network connection.
+It looks feasible. However, it doesn't provide what we wanted, which was that clients get local workers if possible and remote workers only if it's better than waiting. Also workers will signal "ready" to both brokers and can get two jobs at once, while other workers remain idle. It seems this design fails because again we're putting routing logic at the edges.
-* It feels like it would scale smoothly. Interconnecting three or more brokers doesn't get overly complex. If we find this to be a problem, it's easy to solve by adding a super-broker.
+So, idea #2 then. We interconnect the brokers and don't touch the clients or workers, which are REQs like we're used to[figure].
[[code type="textdiagram" title="Idea 2: Brokers Talking to Each Other"]]
Cluster 1 : Cluster 2
@@ -861,6 +849,16 @@ In effect it is just a more sophisticated routing algorithm: brokers become subc
'---' '---' '---' : '---' '---' '---'
[[/code]]
+This design is appealing because the problem is solved in one place, invisible to the rest of the world. Basically, brokers open secret channels to each other and whisper, like camel traders, "Hey, I've got some spare capacity. If you have too many clients, give me a shout and we'll deal".
+
+In effect it is just a more sophisticated routing algorithm: brokers become subcontractors for each other. There are other things to like about this design, even before we play with real code:
+
+* It treats the common case (clients and workers on the same cluster) as default and does extra work for the exceptional case (shuffling jobs between clusters).
+
+* It lets us use different message flows for the different types of work. That means we can handle them differently, e.g., using different types of network connection.
+
+* It feels like it would scale smoothly. Interconnecting three or more brokers doesn't get overly complex. If we find this to be a problem, it's easy to solve by adding a super-broker.
+
We'll now make a worked example. We'll pack an entire cluster into one process. That is obviously not realistic, but it makes it simple to simulate, and the simulation can accurately scale to real processes. This is the beauty of 0MQ--you can design at the micro-level and scale that up to the macro-level. Threads become processes, and then become boxes and the patterns and logic remain the same. Each of our "cluster" processes contains client threads, worker threads, and a broker thread.
We know the basic model well by now:
@@ -873,6 +871,8 @@ We know the basic model well by now:
There are several possible ways to interconnect brokers. What we want is to be able to tell other brokers, "we have capacity", and then receive multiple tasks. We also need to be able to tell other brokers, "stop, we're full". It doesn't need to be perfect; sometimes we may accept jobs we can't process immediately, then we'll do them as soon as possible.
+The simplest interconnect is //federation//, in which brokers simulate clients and workers for each other. We would do this by connecting our frontend to the other broker's backend socket[figure]. Note that it is legal to both bind a socket to an endpoint and connect it to other endpoints.
+
[[code type="textdiagram" title="Cross-connected Brokers in Federation Model"]]
Cluster 1 : Cluster 2
:
@@ -893,8 +893,6 @@ There are several possible ways to interconnect brokers. What we want is to be a
:
[[/code]]
-The simplest interconnect is //federation//, in which brokers simulate clients and workers for each other. We would do this by connecting our frontend to the other broker's backend socket[figure]. Note that it is legal to both bind a socket to an endpoint and connect it to other endpoints.
-
This would give us simple logic in both brokers and a reasonably good mechanism: when there are no clients, tell the other broker "ready", and accept one job from it. The problem is also that it is too simple for this problem. A federated broker would be able to handle only one task at a time. If the broker emulates a lock-step client and worker, it is by definition also going to be lock-step, and if it has lots of available workers they won't be used. Our brokers need to be connected in a fully asynchronous fashion.
The federation model is perfect for other kinds of routing, especially service-oriented architectures (SOAs), which route by service name and proximity rather than load balancing or round robin. So don't dismiss it as useless, it's just not right for all use cases.
@@ -909,6 +907,26 @@ And there is also the flow of information between a broker and its local clients
+++ The Naming Ceremony
+Three flows x two sockets for each flow = six sockets that we have to manage in the broker. Choosing good names is vital to keeping a multisocket juggling act reasonably coherent in our minds. Sockets //do// something and what they do should form the basis for their names. It's about being able to read the code several weeks later on a cold Monday morning before coffee, and not feel any pain.
+
+Let's do a shamanistic naming ceremony for the sockets. The three flows are:
+
+* A //local// request-reply flow between the broker and its clients and workers.
+* A //cloud// request-reply flow between the broker and its peer brokers.
+* A //state// flow between the broker and its peer brokers.
+
+Finding meaningful names that are all the same length means our code will align nicely. It's not a big thing, but attention to details helps. For each flow the broker has two sockets that we can orthogonally call the //frontend// and //backend//. We've used these names quite often. A frontend receives information or tasks. A backend sends those out to other peers. The conceptual flow is from front to back (with replies going in the opposite direction from back to front).
+
+So in all the code we write for this tutorial, we will use these socket names:
+
+* //localfe// and //localbe// for the local flow.
+* //cloudfe// and //cloudbe// for the cloud flow.
+* //statefe// and //statebe// for the state flow.
+
+For our transport and because we're simulating the whole thing on one box, we'll use {{ipc}} for everything. This has the advantage of working like {{tcp}} in terms of connectivity (i.e., it's a disconnected transport, unlike {{inproc}}), yet we don't need IP addresses or DNS names, which would be a pain here. Instead, we will use {{ipc}} endpoints called //something//-{{local}}, //something//-{{cloud}}, and //something//-{{state}}, where //something// is the name of our simulated cluster.
+
+You might be thinking that this is a lot of work for some names. Why not call them s1, s2, s3, s4, etc.? The answer is that if your brain is not a perfect machine, you need a lot of help when reading code, and we'll see that these names do help. It's easier to remember "three flows, two directions" than "six different sockets"[figure].
+
[[code type="textdiagram" title="Broker Socket Arrangement"]]
#---------# #---------# #---------#
| Client | | Broker | | Broker |
@@ -945,37 +963,12 @@ And there is also the flow of information between a broker and its local clients
#---------# #---------# #---------#
[[/code]]
-Three flows x two sockets for each flow = six sockets that we have to manage in the broker. Choosing good names is vital to keeping a multisocket juggling act reasonably coherent in our minds. Sockets //do// something and what they do should form the basis for their names. It's about being able to read the code several weeks later on a cold Monday morning before coffee, and not feel any pain.
-
-Let's do a shamanistic naming ceremony for the sockets. The three flows are:
-
-* A //local// request-reply flow between the broker and its clients and workers.
-* A //cloud// request-reply flow between the broker and its peer brokers.
-* A //state// flow between the broker and its peer brokers.
-
-Finding meaningful names that are all the same length means our code will align nicely. It's not a big thing, but attention to details helps. For each flow the broker has two sockets that we can orthogonally call the //frontend// and //backend//. We've used these names quite often. A frontend receives information or tasks. A backend sends those out to other peers. The conceptual flow is from front to back (with replies going in the opposite direction from back to front).
-
-So in all the code we write for this tutorial, we will use these socket names:
-
-* //localfe// and //localbe// for the local flow.
-* //cloudfe// and //cloudbe// for the cloud flow.
-* //statefe// and //statebe// for the state flow.
-
-For our transport and because we're simulating the whole thing on one box, we'll use {{ipc}} for everything. This has the advantage of working like {{tcp}} in terms of connectivity (i.e., it's a disconnected transport, unlike {{inproc}}), yet we don't need IP addresses or DNS names, which would be a pain here. Instead, we will use {{ipc}} endpoints called //something//-{{local}}, //something//-{{cloud}}, and //something//-{{state}}, where //something// is the name of our simulated cluster.
-
-You might be thinking that this is a lot of work for some names. Why not call them s1, s2, s3, s4, etc.? The answer is that if your brain is not a perfect machine, you need a lot of help when reading code, and we'll see that these names do help. It's easier to remember "three flows, two directions" than "six different sockets"[figure].
-
Note that we connect the cloudbe in each broker to the cloudfe in every other broker, and likewise we connect the statebe in each broker to the statefe in every other broker.
+++ Prototyping the State Flow
Because each socket flow has its own little traps for the unwary, we will test them in real code one-by-one, rather than try to throw the whole lot into code in one go. When we're happy with each flow, we can put them together into a full program. We'll start with the state flow[figure].
-Here is how this works in code:
-
-[[code type="example" title="Prototype state flow" name="peering1"]]
-[[/code]]
-
[[code type="textdiagram" title="The State Flow"]]
#---------#
| Broker |
@@ -1012,6 +1005,11 @@ Here is how this works in code:
#---------#
[[/code]]
+Here is how this works in code:
+
+[[code type="example" title="Prototype state flow" name="peering1"]]
+[[/code]]
+
Notes about this code:
* Each broker has an identity that we use to construct {{ipc}} endpoint names. A real broker would need to work with TCP and a more sophisticated configuration scheme. We'll look at such schemes later in this book, but for now, using generated {{ipc}} names lets us ignore the problem of where to get TCP/IP addresses or names.
@@ -1040,6 +1038,8 @@ If we wanted to send state messages at precise intervals, we'd create a child th
+++ Prototyping the Local and Cloud Flows
+Let's now prototype at the flow of tasks via the local and cloud sockets[figure]. This code pulls requests from clients and then distributes them to local workers and cloud peers on a random basis.
+
[[code type="textdiagram" title="The Flow of Tasks"]]
#---------# #---------#
| Client | | Broker |
@@ -1076,8 +1076,6 @@ If we wanted to send state messages at precise intervals, we'd create a child th
#---------# #---------#
[[/code]]
-Let's now prototype at the flow of tasks via the local and cloud sockets[figure]. This code pulls requests from clients and then distributes them to local workers and cloud peers on a random basis.
-
Before we jump into the code, which is getting a little complex, let's sketch the core routing logic and break it down into a simple yet robust design.
We need two queues, one for requests from local clients and one for requests from cloud clients. One option would be to pull messages off the local and cloud frontends, and pump these onto their respective queues. But this is kind of pointless because 0MQ sockets //are// queues already. So let's use the 0MQ socket buffers as queues.
View
4 chapter8.txt
@@ -386,7 +386,7 @@ The name and slogan first. The trademarks of the 21st century are domain names.
I'm somewhat shy about pushing new projects into the 0MQ community too aggressively, and normally would start a project in either my personal account or the iMatix organization. But we've learned that moving projects after they become popular is counterproductive. My predictions of a future filled with moving pieces are either valid or wrong. If this chapter is valid, we might as well launch this as a 0MQ project from the start. If it's wrong, we can delete the repository later or let it sink to the bottom of a long list of forgotten starts.
-Start with the basics. The protocol (UDP and 0MQ/TCP) will be ZRE (ZeroMQ Realtime Exchange protocol) and the project will be Zyre. I need a second maintainer, so I invite my friend Dong Min (the Korean hacker behind JeroMQ, a pure-Java 0MQ stack) to join. He's been working on very similar ideas so is enthusiastic. We discuss this and we get the idea of building Zyre on top of JeroMQ, as well as on top of CZMQ and libzmq. This would make it a lot easier to run Zyre on Android. It would also give us two fully separate implementations from the start, which is always a good thing for a protocol.
+Start with the basics. The protocol (UDP and 0MQ/TCP) will be ZRE (ZeroMQ Realtime Exchange protocol) and the project will be Zyre. I need a second maintainer, so I invite my friend Dong Min (the Korean hacker behind JeroMQ, a pure-Java 0MQ stack) to join. He's been working on very similar ideas so is enthusiastic. We discuss this and we get the idea of building Zyre on top of JeroMQ, as well as on top of CZMQ and {{libzmq}}. This would make it a lot easier to run Zyre on Android. It would also give us two fully separate implementations from the start, which is always a good thing for a protocol.
So we take the FileMQ project I built in [#advanced-architecture] as a template for a new GitHub project. The GNU autoconf tools are quite decent, but have a painful syntax. It's easiest to copy existing project files and modify them. The FileMQ project builds a library, has test tools, license files, man pages, and so on. It's not too large so it's a good starting point.
@@ -815,7 +815,7 @@ You'll also see that there's no risk of confusion, no way for commands from two
When you are satisfied that this works, we're ready to move on. This version is tagged in the repository as v0.3.0 and you can [https://github.com/zeromq/zyre/tags download the tarball] if you want to check what the code looked like at this stage.
-Note that doing heavy simulation of lots of nodes will probably cause your process to run out of file handles, giving an assertion failure in libzmq. I raised the per-process limit to 30,000 by running (on my Linux box):
+Note that doing heavy simulation of lots of nodes will probably cause your process to run out of file handles, giving an assertion failure in {{libzmq}}. I raised the per-process limit to 30,000 by running (on my Linux box):
[[code]]
ulimit -n 30000
Please sign in to comment.
Something went wrong with that request. Please try again.