Permalink
Browse files

added section on RELAY

  • Loading branch information...
1 parent 34dd381 commit 2fc53cd5109a07a9c84db885f716c520c710dee2 Bela Ban committed Nov 30, 2010
Showing with 97 additions and 6 deletions.
  1. +89 −6 doc/manual/en/modules/advanced.xml
  2. +8 −0 doc/manual/en/modules/protocols.xml
@@ -1592,22 +1592,105 @@
</section>
</section>
- <section>
- <title>Bridging remote clusters</title>
+ <section id="RelayAdvanced">
+ <title>Bridging between remote clusters</title>
<para>
- In 2.12, the RELAY protocol was added to JGroups. It allows for bridging of remote clusters. For example, if
- we have a cluster in New York (NYC) and another one in San Francisco (SFO), then RELAY allows us to bridge
- NYC and SFO, so that multicast messages sent in NYC will be forwarded to SFO and vice versa.
+ In 2.12, the RELAY protocol was added to JGroups (for the properties see <xref linkend="RELAY">RELAY</xref>).
+ It allows for bridging of remote clusters. For example, if we have a cluster in New York (NYC) and another
+ one in San Francisco (SFO), then RELAY allows us to bridge NYC and SFO, so that multicast messages sent in
+ NYC will be forwarded to SFO and vice versa.
</para>
<para>
The NYC and SFO clusters could for example use IP multicasting (UDP as transport), and the bridge could use
- TCP as transport. The SFO and NYC clusters don't even need to be named the same.
+ TCP as transport. The SFO and NYC clusters don't even need to use the same cluster name.
+ </para>
+ <para>
+ <xref linkend="RelayFig"/> shows how the two clusters are bridged.
</para>
<para>
<figure id="RelayFig"><title>Relaying between different clusters</title>
<graphic fileref="images/RELAY.png" format="PNG" align="left" width="15cm"/>
</figure>
+ </para>
+ <para>
+ The cluster on the left side with nodes A (the coordinator), B and C is called "NYC" and use IP
+ multicasting (UDP as transport). The cluster on the right side ("SFO") has nodes D (coordinator), E and F.
+ </para>
+ <para>
+ The bridge between the local clusters NYC and SFO is essentially another cluster with the coordinators
+ (A and D) of the local clusters as members. The bridge typically uses TCP as transport, but any of the
+ supported JGroups transports could be used (including UDP, if supported across a WAN, for instance).
+ </para>
+ <para>
+ Only a coordinator relays traffic between the local and remote cluster. When A crashes or leaves, then the
+ next-in-line (B) takes over and starts relaying.
+ </para>
+ <para>
+ Relaying is done via the RELAY protocol added to the top of the stack. The bridge is configured with
+ the bridge_props property, e.g. bridge_props="/home/bela/tcp.xml". This creates a JChannel inside RELAY.
+ </para>
+ <para>
+ The design is described in detail in JGroups/doc/design/RELAY.txt (part of the source distribution). In
+ a nutshell, multicast messages received in a local cluster are wrapped and forwarded to the remote cluster
+ by a relay (= the coordinator of a local cluster). When a remote cluster receives such a message, it is
+ unwrapped and put onto the local cluster.
+ </para>
+ <para>
+ JGroups uses proxy addresses to wrap an address from a remote cluster into a local address, e.g. when C
+ multicasts a message in the NYC cluster, A will forward it to D, which will re-broadcast the message on
+ its local cluster, with the sender (C) being wrapped in a ProxyAddress(D,C). This means that the sender
+ of the local broadcast will appear as D (so all retransmit requests got to D), but the original sender C
+ is preserved in the proxy address. When node F wants to reply to the sender of the multicast, the destination
+ of the message will be Proxy(D,C), which is intercepted by the RELAY protocol and forwarded to the current
+ relay (D). D then picks the correct destination (C) and sends the message to the remote cluster, where
+ A makes sure C (the original sender) receives it.
+ </para>
+ <para>
+ An important design goal of RELAY is to be able to have completely autonomous clusters, so NYC doesn't for
+ example have to block waiting for credits from SFO, or a node in the SFO cluster doesn't have to ask a node
+ in NYC for retransmission of a missing message.
+ </para>
+ <section>
+ <title>Views</title>
+ <para>
+ RELAY tries to present a <emphasis>global view</emphasis> to the application, e.g. a view received by
+ nodes could be {D,E,F,A,B,C}. This view is the same on all nodes, and a global view is generated by
+ taking the two local views, e.g. A|5 {A,B,C} and D|2 {D,E,F}, comparing the coordinators' addresses
+ (the UUIDs for A and D) and concatenating the views into a list. So if D's UUID is greater than
+ A's UUID, we first add D's members into the global view ({D,E,F}), and then A's members.
+ </para>
+ <para>
+ Therefore, we'll always see all of A's members, followed by all of D's members, or the other way round.
+ </para>
+ <para>
+ To see which nodes are local and which ones remote, we can iterate through the addresses and check
+ for ProxyAddress (via instanceof). For example, if we denote remoteness by the suffix 'p', then global
+ view {D,E,F,A,B,C} could be A|5 {Dp,Ep,Fp,A,B,C} in NYC and D|2 {D,E,F,Ap,Bp,Cp} in SFO.
+ </para>
+ </section>
+ <section>
+ <title>Configuration</title>
+ <para>
+ To setup a relay, we need essentially 3 XML configuration files: 2 to configure the local clusters and
+ 1 for the bridge.
+ </para>
+ <para>
+ To configure the first local cluster, we can copy udp.xml from the JGroups distribution and add RELAY on top
+ of it: &lt;RELAY bridge_props="/home/bela/tcp.xml" /&gt;. Let's say we call this config relay.xml.
+ </para>
+ <para>
+ The second local cluster can be configured by copying relay.xml to relay2.xml. Then change the
+ mcast_addr and/or mcast_port, so we actually have 2 different cluster in case we run instances of
+ both clusters in the same network. Of course, if the nodes of one cluster are run in a different
+ network from the nodes of the other cluster, and they cannot talk to each other, then we can simply
+ use the same configuration.
</para>
+ <para>
+ The bridge is configured by taking the stock tcp.xml and making sure both local clusters can see each
+ other through TCP.
+ </para>
+ </section>
+
</section>
<section>
@@ -961,6 +961,14 @@
${SCOPE}
</section>
+ <section id="RELAY">
+ <title>RELAY</title>
+ <para>
+ RELAY bridges traffic between seperate clusters, see <xref linkend="RelayAdvanced"/> for details.
+ </para>
+ ${RELAY}
+ </section>
+
<section id="STOMP">
<title>STOMP</title>
<para>

0 comments on commit 2fc53cd

Please sign in to comment.