Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion source/carbonio/admincli/toc.rst
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ mandatory.
carboniobackup
mobile
storages
activereplica
usermailreplica
videoserver
custom
mesh
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
.. _ureplica:

.. _activereplica:

=================
Expand Down Expand Up @@ -54,6 +56,8 @@ There are **two requirements** to satisfy to be able to install the
#. All the primary volumes of the mailbox **must be configured** as
:ref:`Centralized Storage <pws_centralized_storage>`.

.. _enable-ur:

Enabling |ur|
=============

Expand Down Expand Up @@ -85,12 +89,14 @@ To verify that the settings have been applied and the service operates
correctly, you can use the commands presented in section :ref:`ar-ts`
below.

.. _use-ur:

|ur| Usage
==========

A number of CLI commands can be used to carry out routine operations
with the |ur|: :ref:`initialise <ar-init>`, :ref:`monitor
<ar-monit>`, :ref:`promote <ar-promo>`, and :ref:`delete <ar-del>` a
with the |ur|: :ref:`initialise <ur-init>`, :ref:`monitor
<ur-monit>`, :ref:`promote <ur-promo>`, and :ref:`delete <ur-del>` a
|ur|.

Limitations of the Commands
Expand All @@ -104,7 +110,7 @@ The command presented in this section **do not support**:
* distribution lists


.. _ar-init:
.. _ur-init:

|ur| Initialisation
----------------------
Expand Down Expand Up @@ -142,7 +148,7 @@ Example of valid commands are:
The Global Administrator will receive a notification as soon as the
|ur| initialisation is completed.

.. _ar-monit:
.. _ur-monit:

|ur| Monitoring
------------------
Expand Down Expand Up @@ -197,7 +203,7 @@ Then, for each replica:
priority prority of the replica
canConnectToKafka status of connection to kafka

.. _ar-promo:
.. _ur-promo:

|ur| Promotion
--------------
Expand Down Expand Up @@ -243,7 +249,7 @@ Example of valid commands are:

Global Admin will receive a notification as soon as the |ur| promotion is completed.

.. _ar-del:
.. _ur-del:

|ur| Deletion
-------------
Expand Down
5 changes: 2 additions & 3 deletions source/carbonio/architecture/components.rst
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@ This is the list of Components that make up a |product| installation.

When installing the Components, be careful that:

* Postgres, Grafana, Zookeper, and Kafka are third-party software that
are installed from their respective official repositories
* Postgres, Grafana, and Kafka are third-party software that are
installed from their respective official repositories

* On each Node, you need to install package
``service-discover-agent``, except on the Node on which
Expand Down Expand Up @@ -211,7 +211,6 @@ When installing the Components, be careful that:

Packages:

* ZooKeeper
* Apache Kafka

.. grid-item-card:: Directory Replica
Expand Down
158 changes: 10 additions & 148 deletions source/carbonio/architecture/components/component-es.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,156 +5,18 @@ Event Streaming

This Component is required to enable the |carbonio|
:ref:`activereplica` feature, the foundation of High Availability on
|product|, and is based on Apache's *Kafka* and *ZooKeeper*, which
must be installed together on the same Node. For better performances,
it is strongly suggested to install both the services on a dedicated
Node, otherwise you can opt to directly access a Kafka + ZooKeeper
service provided as a Saas solution.
|product|, and is based on Apache's *Kafka*, which must be installed
together on the same Node. For better performances, it is strongly
suggested to install both the services on a dedicated Node, otherwise
you can opt to directly access a Kafka service provided as
a Saas solution.

In the remainder, you find the installation and configuration
instructions for both software on a node dedicated in the same
infrastructure that hosts the other |product| Nodes.

It is possible to install multiple Event Streaming Components in a
|product| infrastructure: in this case, you need to make sure that the
ZooKeeper's configuration is replicated on every Event Streaming Node.

Installation of Apache ZooKeeper
--------------------------------

Instructions are currently provided for **Ubuntu**; installation on
*RHEL* should be almost the same, but official directions will follow
soon.

Requirements and Preliminary Tasks
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

All the installation tasks are done via CLI, so you need some
acquaintance with it.

We also assume you are using ZooKeeper version **3.8.3**, you can
however use other **3.8.x** versions: check their availability on the
`Apache ZooKeeper <https://zookeeper.apache.org/releases.html>`_
official page, and remember to replace the version in the directions
below.

Moreover, to setup ZooKeeper, you need:

* a dedicated user, ``carbonio-queue``, shared with *Kafka*

.. code:: console

# adduser --system --group --home '/var/lib/queue/' 'carbonio-queue'

* a ZooKeeper data directory: :file:`/var/lib/queue/zookeeper/`

.. code:: console

# mkdir /var/lib/queue/zookeeper/


Installation
~~~~~~~~~~~~

.. code:: console

# apt update -y
# apt install -y openjdk-17-jre-headless
# wget https://dlcdn.apache.org/zookeeper/zookeeper-3.8.3/apache-zookeeper-3.8.3-bin.tar.gz
# tar -xpvzf apache-zookeeper-3.8.3-bin.tar.gz -C /opt/
# mv /opt/apache-zookeeper-3.8.3-bin /opt/zookeeper
# rm -f apache-zookeeper.tar.gz
# chown carbonio-queue:root -R /opt/zookeeper/
# chown carbonio-queue: -R /var/lib/queue/

Configuration
~~~~~~~~~~~~~

Copy the sample configuration into :file:`/opt/zookeeper/conf/zoo.cfg` and
update the ``dataDir``:

.. code:: console

# cp /opt/zookeeper/conf/zoo_sample.cfg /opt/zookeeper/conf/zoo.cfg
# sed -i -e 's@dataDir=.*@dataDir=/var/lib/queue/zookeeper/@g' /opt/zookeeper/conf/zoo.cfg

.. hint:: If you do not feel comfortable with the :command:`sed`
command, you can edit the file manually: find the line starting
with ``dataDir=`` and replace any path after the **=** with
``/var/lib/queue/zookeeper``.

Define a ZooKeeper ID
~~~~~~~~~~~~~~~~~~~~~

Each Node in a |product| infrastructure **must have** a unique
ZooKeeper ID, which is required for its correct operating.

Define a value for the Node (we use *10*) on which ZooKeper is
installed and add it to the file

.. code:: console

# ZOOKEEPER_ID=10
# echo $ZOOKEEPER_ID > /var/lib/queue/zookeeper/myid

If you install a second and even a third Event Streaming Component, you
have then to append at the end of the configuration file
:file:`/opt/zookeeper/conf/zoo.cfg` one entry for every zookeeper
node, using the format ``server.[ZOOKEEPER_ID]=[NODE_IP]:2888:3888``

For example, suppose you want to install a three-nodes Event
Streaming. You have already you assigned ``ZOOKEEPER_ID=1O`` to node
IP ``10.0.10.11`` and you add

* ID ``ZOOKEEPER_ID=2O`` to node IP ``10.0.10.12``
* ID ``ZOOKEEPER_ID=3O`` to node IP ``10.0.10.13``

You need to make sure that three entries are added to file
:file:`/opt/zookeeper/conf/zoo.cfg` **on every Node**:

.. code:: console

# echo "server.10=10.0.10.11:2888:3888" >> /opt/zookeeper/conf/zoo.cfg
# echo "server.20=10.0.10.12:2888:3888" >> /opt/zookeeper/conf/zoo.cfg
# echo "server.30=10.0.10.13:2888:3888" >> /opt/zookeeper/conf/zoo.cfg

Create System Unit file
~~~~~~~~~~~~~~~~~~~~~~~

Copy this snippet to define a minimal zookeeper configuration into
file :file:`/lib/systemd/system/carbonio-zookeeper.service`.

.. code:: ini

[Unit]
Description=ZooKeeper Service
Documentation=http://zookeeper.apache.org
Requires=network.target
After=network.target

[Service]
Type=forking
Restart=on-failure
RestartSec=15
LimitNOFILE=65536
User=carbonio-queue
Group=carbonio-queue
ExecStart=/opt/zookeeper/bin/zkServer.sh start /opt/zookeeper/conf/zoo.cfg
ExecStop=/opt/zookeeper/bin/zkServer.sh stop /opt/zookeeper/conf/zoo.cfg
ExecReload=/opt/zookeeper/bin/zkServer.sh restart /opt/zookeeper/conf/zoo.cfg
WorkingDirectory=/var/lib/queue/zookeeper/

[Install]
WantedBy=default.target

Enable ZooKeeper service
~~~~~~~~~~~~~~~~~~~~~~~~

.. code:: console

# systemctl daemon-reload
# systemctl enable carbonio-zookeeper
# systemctl start carbonio-zookeeper.service
|product| infrastructure.

Installation of Apache Kafka
----------------------------
Expand All @@ -167,9 +29,9 @@ use other **3.1.x** versions: check their availability on the `Apache
Kafka <https://kafka.apache.org/downloads>`_ official page, and
remember to replace the version in the directions below.

Moreover, to setup ZooKeeper, you need:
Moreover, to setup Apache Kafka, you need:

* a dedicated user, ``carbonio-queue``, which is the same used by ZooKeeper
* a dedicated user, ``carbonio-queue``

* Kafka data directory: :file:`/var/lib/queue/kafka/logs/`

Expand Down Expand Up @@ -227,8 +89,8 @@ infrastructure, ``log.dirs``, and ``topic`` in files
Create System Unit File
~~~~~~~~~~~~~~~~~~~~~~~

Copy this snippet to define a minimal zookeeper configuration into
file :file:`/lib/systemd/system/carbonio-kafka.service`.
Copy this snippet to define a minimal configuration into file
:file:`/lib/systemd/system/carbonio-kafka.service`.

.. code:: ini

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,12 +18,6 @@ These are the commands to be issued as the |ru|.

# ssh root@mob-ha-svcs1.demo.zextras.io -L8550:127.0.0.1:8500

#. Check zookeeper service status

.. code:: console

# systemctl status carbonio-zookeeper.service

#. Check Kafka services status

.. code:: console
Expand All @@ -38,6 +32,12 @@ These are the commands to be issued as the |ru|.

These are the commands to be issued as the |zu|.

#. Check patroni service, including finding the leader

.. code:: console

# sytemctl -u patroni -f

#. Get Global Broker info

.. code:: console
Expand Down

This file was deleted.

Loading