Skip to content

Commit c572dc6

Browse files
liviuchircurazvancrainea
liviuchircu
authored and
razvancrainea
committed
clusterer/ratelimit: Document the new Bridge-Replication feature
1 parent 344688c commit c572dc6

3 files changed

Lines changed: 194 additions & 1 deletion

File tree

‎db/schema/opensips-clusterer.xml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,5 +9,5 @@
99
<database xmlns:xi="http://www.w3.org/2001/XInclude">
1010
<name>Clusterer support</name>
1111
<xi:include href="clusterer.xml"/>
12-
<xi:include href="clusterer_link.xml"/>
12+
<xi:include href="clusterer_bridge.xml"/>
1313
</database>

‎modules/clusterer/doc/clusterer_admin.xml‎

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,87 @@
4343
</para>
4444
</section>
4545

46+
<section id="bridge_replication" xreflabel="Cluster-Bridge Replication">
47+
<title>Cluster-Bridge Replication</title>
48+
<para>
49+
<emphasis>(added in OpenSIPS 4.0)</emphasis>
50+
</para>
51+
<para>
52+
Cluster-Bridge Replication (or "bridge replication") allows modules to
53+
exchange data across <emphasis>different</emphasis> clusters. This
54+
is meant to serve as a topology/data flow optimization feature, and it
55+
could be useful in some OpenSIPS cluster setups with multiple data
56+
centers. In such cases, it might be desirable to minimize the amount
57+
of inter-DC replication channels, for example:
58+
</para>
59+
<para>
60+
<programlisting format="linespecific">
61+
62+
Before (standard, full-mesh replication):
63+
64+
DC #1 DC #2
65+
WAN link
66+
A &lt;─────────> C
67+
^ \ / ^
68+
│ \ / │ 4 x OpenSIPS nodes, 1 x cluster
69+
│ X │ (8 inter-DC replication channels)
70+
│ / \ │ AC, AD, BC, BD, CA, CB, DA, DB
71+
v / \ v
72+
B &lt;─────────> D
73+
WAN link
74+
cluster_id: 1
75+
76+
After (cluster-bridged replication):
77+
78+
DC #1 DC #2
79+
WAN link
80+
A ──────────> C
81+
^ ^
82+
│ │ 4 x OpenSIPS nodes, 2 x clusters
83+
│ │ (2 inter-DC replication channels)
84+
│ │ AC, DB
85+
v v
86+
B &lt;────────── D
87+
WAN link
88+
cluster_id: 1 cluster_id: 2
89+
sender: <emphasis role='bold'>wan1</emphasis> shtag sender: <emphasis role='bold'>wan1</emphasis> shtag
90+
</programlisting>
91+
92+
</para>
93+
<para>
94+
A <emphasis>new table</emphasis> has been added to represent the
95+
inter-cluster replication bridges, named
96+
<link linkend="param_db_bridge_table">clusterer_bridge</link>:
97+
<programlisting>
98+
99+
mysql> select * from clusterer_bridge;
100+
+----+-----------+-----------+------------+-------------------------------+
101+
| id | cluster_a | cluster_b | send_shtag | dst_node_csv |
102+
+----+-----------+-----------+------------+-------------------------------+
103+
| 1 | 1 | 2 | wan1 | bin:10.0.0.213,bin:10.0.0.214 |
104+
| 2 | 2 | 1 | wan1 | bin:10.0.0.210,bin:10.0.0.212 |
105+
+----+-----------+-----------+------------+-------------------------------+
106+
2 rows in set (0,00 sec)
107+
</programlisting>
108+
<emphasis>Example of a bi-directional bridge between clusters "1" and "2".</emphasis>
109+
</para>
110+
<para>
111+
The "send_shtag" controls the originator node for each cluster bridge defined in the table.
112+
Only the node with the "active" tag will actually send data over the network.
113+
Sharing tags can be defined using the <xref linkend="param_sharing_tag"/> module parameter.
114+
</para>
115+
<para>
116+
The "dst_node_csv" functions as a list of remote cluster nodes to try.
117+
The module will attempt a single TCP send per node, in failover fashion (always same order).
118+
</para>
119+
120+
<para>
121+
At the time of writing, the only module using the new bridge replication
122+
feature is <ulink url='ratelimit#bridge_replication'>ratelimit</ulink>,
123+
in order to optimize its "CPS pipes broadcasting" replication mechanism.
124+
</para>
125+
</section>
126+
46127
<section id="dependencies" xreflabel="Dependencies">
47128
<title>Dependencies</title>
48129
<section>
@@ -170,6 +251,26 @@ modparam("clusterer", "db_url",
170251
<programlisting format="linespecific">
171252
...
172253
modparam("clusterer", "db_table", "clusterer")
254+
...
255+
</programlisting>
256+
</example>
257+
</section>
258+
259+
<section id="param_db_bridge_table" xreflabel="db_bridge_table">
260+
<title><varname>db_bridge_table</varname></title>
261+
<para>
262+
The name of the table storing the inter-cluster bridge definitions.
263+
</para>
264+
<para>
265+
<emphasis>
266+
Default value is <quote>clusterer_bridge</quote>.
267+
</emphasis>
268+
</para>
269+
<example>
270+
<title>Set <varname>db_bridge_table</varname> parameter</title>
271+
<programlisting format="linespecific">
272+
...
273+
modparam("clusterer", "db_bridge_table", "clusterer_bridge")
173274
...
174275
</programlisting>
175276
</example>

‎modules/ratelimit/doc/ratelimit_admin.xml‎

Lines changed: 92 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@
3636
will replicate through bin/clusterer.
3737
</para>
3838
</section>
39+
3940
<section>
4041
<title>Use Cases</title>
4142
<para>
@@ -72,6 +73,7 @@
7273
response.
7374
</para>
7475
</section>
76+
7577
<section>
7678
<title>Static Rate Limiting Algorithms</title>
7779
<para>
@@ -140,6 +142,7 @@
140142
</para>
141143
</section>
142144
</section>
145+
143146
<section>
144147
<title>Dynamic Rate Limiting Algorithms</title>
145148
<para>
@@ -186,6 +189,30 @@
186189
</para>
187190
</section>
188191
</section>
192+
193+
<section id="bridge_replication">
194+
<title>Cluster-Bridge Replication</title>
195+
<para>
196+
<emphasis>(added in OpenSIPS 4.0)</emphasis>
197+
</para>
198+
<para>
199+
Cluster-bridge replication can be useful in some specific scenarios
200+
where you have multiple clusters of OpenSIPS servers, geographically
201+
spread across the country or beyond. In this case, as an improvement
202+
to the standard, full-mesh replication strategy, you can alternatively
203+
group them into multiple cluster IDs based on geo-location, then define
204+
the replication "bridges" between them. Full details in the
205+
<ulink url='clusterer#bridge_replication'>cluster-bridge replication</ulink>
206+
chapter of the clusterer module.
207+
</para>
208+
<para>
209+
At ratelimit level, you only need to enable <xref linkend="param_bridge_replication"/>,
210+
after which the CPS pipes should start flowing to remote clusters. Make
211+
sure to tweak <xref linkend="param_bridge_repl_timer_interval"/> and
212+
<xref linkend="param_bridge_repl_timer_expire"/> as needed.
213+
</para>
214+
</section>
215+
189216
<section id="dependencies" xreflabel="Dependencies">
190217
<title>Dependencies</title>
191218
<section>
@@ -449,6 +476,71 @@ modparam("ratelimit", "repl_timer_expire", 10)
449476
...
450477
modparam("ratelimit", "pipe_replication_cluster", 1)
451478
...
479+
</programlisting>
480+
</example>
481+
</section>
482+
<section id="param_bridge_replication" xreflabel="bridge_replication">
483+
<title><varname>bridge_replication</varname> (boolean)</title>
484+
<para>
485+
Enable the <ulink url='clusterer#bridge_replication'>cluster-bridge replication</ulink>
486+
feature, if applicable (e.g. the current <xref linkend="param_pipe_replication_cluster"/> has
487+
at least one bridge definition to a foreign cluster).
488+
</para>
489+
<para>
490+
<emphasis>
491+
Default value is <emphasis>false</emphasis>. (disabled)
492+
</emphasis>
493+
</para>
494+
<example>
495+
<title>Set <varname>bridge_replication</varname> parameter</title>
496+
<programlisting format="linespecific">
497+
...
498+
modparam("ratelimit", "bridge_replication", true)
499+
...
500+
</programlisting>
501+
</example>
502+
</section>
503+
<section id="param_bridge_repl_timer_interval" xreflabel="bridge_repl_timer_interval">
504+
<title><varname>bridge_repl_timer_interval</varname> (string)</title>
505+
<para>
506+
Timer in milliseconds, used to specify how often the module
507+
should replicate its cluster-local counters to remote clusters,
508+
if <ulink url='clusterer#bridge_replication'>bridged replication</ulink>
509+
is in use, as long as it holds the required sharing tag(s).
510+
</para>
511+
<para>
512+
<emphasis>
513+
Default value is 500 ms.
514+
</emphasis>
515+
</para>
516+
<example>
517+
<title>Set <varname>bridge_repl_timer_interval</varname> parameter</title>
518+
<programlisting format="linespecific">
519+
...
520+
modparam("ratelimit", "bridge_repl_timer_interval", 500)
521+
...
522+
</programlisting>
523+
</example>
524+
</section>
525+
<section id="param_bridge_repl_timer_expire" xreflabel="bridge_repl_timer_expire">
526+
<title><varname>bridge_repl_timer_expire</varname> (string)</title>
527+
<para>
528+
Timer in seconds, used to specify when the counter received
529+
from a different cluster should no longer be taken into account.
530+
This is used to prevent obsolete values, in case an entire cluster
531+
(data center) is down and stops replicating its counters.
532+
</para>
533+
<para>
534+
<emphasis>
535+
Default value is 20 s.
536+
</emphasis>
537+
</para>
538+
<example>
539+
<title>Set <varname>bridge_repl_timer_expire</varname> parameter</title>
540+
<programlisting format="linespecific">
541+
...
542+
modparam("ratelimit", "bridge_repl_timer_expire", 20)
543+
...
452544
</programlisting>
453545
</example>
454546
</section>

0 commit comments

Comments
 (0)