From 8ef78be035ad60356b4b81dc391678089fdb9610 Mon Sep 17 00:00:00 2001 From: Kseniia Antonova Date: Mon, 30 Sep 2024 11:41:26 +0300 Subject: [PATCH 1/5] Add description for the `age` and `confirm_lag` fields --- .../reference_lua/box_info/synchro.rst | 83 +++++++++++-------- 1 file changed, 50 insertions(+), 33 deletions(-) diff --git a/doc/reference/reference_lua/box_info/synchro.rst b/doc/reference/reference_lua/box_info/synchro.rst index 69608fdb8..14df96ac3 100644 --- a/doc/reference/reference_lua/box_info/synchro.rst +++ b/doc/reference/reference_lua/box_info/synchro.rst @@ -1,8 +1,7 @@ .. _box_info_synchro: -================================================================================ box.info.synchro -================================================================================ +================ .. module:: box.info @@ -29,6 +28,13 @@ box.info.synchro With elections enabled, an instance runs ``box.ctl.promote()`` command automatically after winning the elections. To clear the ownership, call :ref:`box.ctl.demote() ` on the synchronous queue owner. + .. note:: + + When Raft election is enabled and :ref:`replication.election_mode ` + is set to ``candidate``, the new Raft leader claims the queue automatically. + It means that the value of ``box.info.synchro.queue.owner`` becomes equal to :ref:`box.info.election.leader `. + When Raft enabled, no manual intervention with ``box.ctl.promote()`` or ``box.ctl.demote()`` is required. + - ``term`` (since version :doc:`2.10.0 `) -- current queue term. It contains the term of the last ``PROMOTE`` request. Usually, it is equal to :ref:`box.info.election.term `. @@ -43,6 +49,12 @@ box.info.synchro Until the request is complete, any other incoming synchronous transactions and system requests will be delayed. + - ``age`` (since version :doc:`3.2.0 `) -- the time in seconds that the oldest entry currently + present in the queue has spent waiting for the quorum to collect. + + - ``confirm_lag`` (since version :doc:`3.2.0 `) -- the time in seconds that the latest successfully + confirmed entry waited for the quorum to collect. + * ``quorum`` -- the resulting value of the :ref:`replication_synchro_quorum ` configuration option. Since version :doc:`2.5.3 `, the option can be set as a dynamic formula. @@ -50,17 +62,19 @@ box.info.synchro **Example 1:** - In this example, the ``quorum`` field is equal to ``1``. + In this example, the ``quorum`` field is equal to `1`. That is, synchronous transactions work like asynchronous ones. `1` means that a successful WAL writing to the master is enough to commit. - .. code-block:: tarantoolsession + .. code-block:: console - tarantool> box.info.synchro + instance001> box.info.synchro --- - queue: owner: 1 + confirm_lag: 0 term: 2 + age: 0 len: 0 busy: false quorum: 1 @@ -68,24 +82,25 @@ box.info.synchro **Example 2:** - First, set a quorum number and a timeout for synchronous replication using the following command: + First, set a quorum number and a timeout for synchronous replication in the configuration file (``config.yaml``): - .. code-block:: tarantoolsession + .. code-block:: yaml - tarantool> box.cfg{ - > replication_synchro_quorum=2, - > replication_synchro_timeout=1000 - > } + replication: + synchro_quorum: 2 + synchro_timeout: 1000 - Next, check the current state of synchronous replication: + Check the current state of synchronous replication: - .. code-block:: tarantoolsession + .. code-block:: console - tarantool> box.info.synchro + app:instance001> box.info.synchro --- - queue: - owner: 1 - term: 2 + owner: 2 + confirm_lag: 0 + term: 28 + age: 0 len: 0 busy: false quorum: 2 @@ -94,51 +109,53 @@ box.info.synchro Create a space called ``sync`` and enable synchronous replication on this space. Then, create an index. - .. code-block:: tarantoolsession + .. code-block:: console - tarantool> s = box.schema.space.create("sync", {is_sync=true}) - tarantool> _ = s:create_index('pk') + app:instance001> s = box.schema.space.create("sync", {is_sync=true}) + app:instance001> _ = s:create_index('pk') After that, use ``box.ctl.promote()`` function to claim a queue: - .. code-block:: tarantoolsession + .. code-block:: console - tarantool> box.ctl.promote() + app:instance001> box.ctl.promote() Next, perform data manipulations: - .. code-block:: tarantoolsession + .. code-block:: console - tarantool> require('fiber').new(function() box.space.sync:replace{1} end) + app:instance001> require('fiber').new(function() box.space.sync:replace{1} end) --- - status: suspended name: lua - id: 119 + id: 127 ... - tarantool> require('fiber').new(function() box.space.sync:replace{1} end) + app:instance001> require('fiber').new(function() box.space.sync:replace{1} end) --- - status: suspended name: lua - id: 120 + id: 128 ... - tarantool> require('fiber').new(function() box.space.sync:replace{1} end) + app:instance001> require('fiber').new(function() box.space.sync:replace{1} end) --- - status: suspended name: lua - id: 121 + id: 129 ... If you call the ``box.info.synchro`` command again, you will see that now there are 3 transactions waiting in the queue: - .. code-block:: tarantoolsession + .. code-block:: console - tarantool> box.info.synchro + app:instance001> box.info.synchro --- - queue: owner: 1 - term: 2 - len: 3 + confirm_lag: 0 + term: 29 + age: 0 + len: 0 busy: false quorum: 2 - ... \ No newline at end of file + ... From 5b8a237b8588d2349c8ae22e8249d9f9d374ec6a Mon Sep 17 00:00:00 2001 From: Kseniia Antonova Date: Wed, 9 Oct 2024 12:09:49 +0300 Subject: [PATCH 2/5] Apply suggestions from code review --- .../reference_lua/box_info/synchro.rst | 35 +++++++++++-------- 1 file changed, 20 insertions(+), 15 deletions(-) diff --git a/doc/reference/reference_lua/box_info/synchro.rst b/doc/reference/reference_lua/box_info/synchro.rst index 14df96ac3..15efe3c1d 100644 --- a/doc/reference/reference_lua/box_info/synchro.rst +++ b/doc/reference/reference_lua/box_info/synchro.rst @@ -25,15 +25,12 @@ box.info.synchro but they can't create any synchronous transactions. To claim or reclaim the queue, use :ref:`box.ctl.promote() ` on the instance that you want to promote. - With elections enabled, an instance runs ``box.ctl.promote()`` command automatically after winning the elections. To clear the ownership, call :ref:`box.ctl.demote() ` on the synchronous queue owner. - .. note:: - - When Raft election is enabled and :ref:`replication.election_mode ` - is set to ``candidate``, the new Raft leader claims the queue automatically. - It means that the value of ``box.info.synchro.queue.owner`` becomes equal to :ref:`box.info.election.leader `. - When Raft enabled, no manual intervention with ``box.ctl.promote()`` or ``box.ctl.demote()`` is required. + When Raft election is enabled and :ref:`replication.election_mode ` + is set to ``candidate``, the new Raft leader claims the queue automatically after winning the elections. + It means that the value of ``box.info.synchro.queue.owner`` becomes equal to :ref:`box.info.election.leader `. + When Raft enabled, no manual intervention with ``box.ctl.promote()`` or ``box.ctl.demote()`` is required. - ``term`` (since version :doc:`2.10.0 `) -- current queue term. It contains the term of the last ``PROMOTE`` request. @@ -97,22 +94,30 @@ box.info.synchro app:instance001> box.info.synchro --- - queue: - owner: 2 + owner: 1 confirm_lag: 0 - term: 28 + term: 2 age: 0 len: 0 busy: false quorum: 2 ... - Create a space called ``sync`` and enable synchronous replication on this space. - Then, create an index. + Create a space called ``sync`` and enable synchronous replication on this space: .. code-block:: console app:instance001> s = box.schema.space.create("sync", {is_sync=true}) + --- + ... + + Then, create an index: + + .. code-block:: console + app:instance001> _ = s:create_index('pk') + --- + ... After that, use ``box.ctl.promote()`` function to claim a queue: @@ -128,19 +133,19 @@ box.info.synchro --- - status: suspended name: lua - id: 127 + id: 130 ... app:instance001> require('fiber').new(function() box.space.sync:replace{1} end) --- - status: suspended name: lua - id: 128 + id: 131 ... app:instance001> require('fiber').new(function() box.space.sync:replace{1} end) --- - status: suspended name: lua - id: 129 + id: 132 ... If you call the ``box.info.synchro`` command again, @@ -153,7 +158,7 @@ box.info.synchro - queue: owner: 1 confirm_lag: 0 - term: 29 + term: 2 age: 0 len: 0 busy: false From 149218ef443a1bf193ae961f67efdea7a03c6d77 Mon Sep 17 00:00:00 2001 From: Kseniia Antonova Date: Fri, 11 Oct 2024 10:34:39 +0300 Subject: [PATCH 3/5] Update text after review, add code snippet --- .../box_info_synchro/README.md | 23 ++++ .../box_info_synchro/config.yaml | 31 +++++ .../box_info_synchro/instances.yml | 2 + .../reference_lua/box_info/synchro.rst | 109 +++++++++--------- 4 files changed, 113 insertions(+), 52 deletions(-) create mode 100644 doc/code_snippets/snippets/replication/instances.enabled/box_info_synchro/README.md create mode 100644 doc/code_snippets/snippets/replication/instances.enabled/box_info_synchro/config.yaml create mode 100644 doc/code_snippets/snippets/replication/instances.enabled/box_info_synchro/instances.yml diff --git a/doc/code_snippets/snippets/replication/instances.enabled/box_info_synchro/README.md b/doc/code_snippets/snippets/replication/instances.enabled/box_info_synchro/README.md new file mode 100644 index 000000000..e4c95b392 --- /dev/null +++ b/doc/code_snippets/snippets/replication/instances.enabled/box_info_synchro/README.md @@ -0,0 +1,23 @@ +# Using box.info.synchro + +A sample application demonstrating how to work with [box.info.synchro](https://www.tarantool.io/en/doc/latest/reference/reference_lua/box_info/synchro/). + +## Running + +To start all instances, execute the following command in the [replication](../../../replication) directory: + +```console +$ tt start box_info_synchro +``` + +To check the instance status, run: + +```console +$ tt status box_info_synchro +``` + +To connect to the ``instance001`` instance, run: + +```console +$ tt connect box_info_synchro:instance001 +``` \ No newline at end of file diff --git a/doc/code_snippets/snippets/replication/instances.enabled/box_info_synchro/config.yaml b/doc/code_snippets/snippets/replication/instances.enabled/box_info_synchro/config.yaml new file mode 100644 index 000000000..06f27a7b7 --- /dev/null +++ b/doc/code_snippets/snippets/replication/instances.enabled/box_info_synchro/config.yaml @@ -0,0 +1,31 @@ +credentials: + users: + replicator: + password: 'topsecret' + roles: [replication] + +iproto: + advertise: + peer: + login: replicator + +groups: + group001: + replicasets: + replicaset001: + instances: + instance001: + database: + mode: rw + replication: + synchro_quorum: 2 + synchro_timeout: 1000 + iproto: + listen: + - uri: '127.0.0.1:3301' + instance002: + database: + mode: ro + iproto: + listen: + - uri: '127.0.0.1:3302' diff --git a/doc/code_snippets/snippets/replication/instances.enabled/box_info_synchro/instances.yml b/doc/code_snippets/snippets/replication/instances.enabled/box_info_synchro/instances.yml new file mode 100644 index 000000000..75e286d69 --- /dev/null +++ b/doc/code_snippets/snippets/replication/instances.enabled/box_info_synchro/instances.yml @@ -0,0 +1,2 @@ +instance001: +instance002: \ No newline at end of file diff --git a/doc/reference/reference_lua/box_info/synchro.rst b/doc/reference/reference_lua/box_info/synchro.rst index 15efe3c1d..85927b553 100644 --- a/doc/reference/reference_lua/box_info/synchro.rst +++ b/doc/reference/reference_lua/box_info/synchro.rst @@ -53,7 +53,7 @@ box.info.synchro confirmed entry waited for the quorum to collect. * ``quorum`` -- the resulting value of the - :ref:`replication_synchro_quorum ` configuration option. + :ref:`replication.synchro_quorum ` configuration option. Since version :doc:`2.5.3 `, the option can be set as a dynamic formula. In this case, the value of the ``quorum`` member depends on the current number of replicas. @@ -63,9 +63,9 @@ box.info.synchro That is, synchronous transactions work like asynchronous ones. `1` means that a successful WAL writing to the master is enough to commit. - .. code-block:: console + .. code-block:: tarantoolsession - instance001> box.info.synchro + box_info_synchro:instance001> box.info.synchro --- - queue: owner: 1 @@ -79,19 +79,48 @@ box.info.synchro **Example 2:** - First, set a quorum number and a timeout for synchronous replication in the configuration file (``config.yaml``): + Example on GitHub: `box_info_synchro `_ - .. code-block:: yaml + In this example, there are two instances: - replication: - synchro_quorum: 2 - synchro_timeout: 1000 + - ``instance001`` is going to be the leader. + - ``instance002`` is a follower instance. + + .. literalinclude:: /code_snippets/snippets/replication/instances.enabled/box_info_synchro/config.yaml + :language: yaml + :start-at: groups + :end-at: 3302 + :dedent: + + On the **first** instance, grant the user with the ``super`` role: + + .. code-block:: tarantoolsession + + box_info_synchro:instance001> box.schema.user.grant('guest', 'super') + + After that, use ``box.ctl.promote()`` function to claim the queue: + + .. code-block:: tarantoolsession + + box_info_synchro:instance001> box.ctl.promote() + + Create a space called ``sync`` and enable synchronous replication on this space: + + .. code-block:: tarantoolsession + + box_info_synchro:instance001> s = box.schema.space.create("sync", {is_sync=true}) + + Then, create an index: + + .. code-block:: tarantoolsession + + box_info_synchro:instance001> _ = s:create_index('pk') Check the current state of synchronous replication: - .. code-block:: console + .. code-block:: tarantoolsession - app:instance001> box.info.synchro + box_info_synchro:instance001> box.info.synchro --- - queue: owner: 1 @@ -103,64 +132,40 @@ box.info.synchro quorum: 2 ... - Create a space called ``sync`` and enable synchronous replication on this space: + On the **second** instance, simulate failure like if this instance would crash or go out of the network: - .. code-block:: console + .. code-block:: tarantoolsession - app:instance001> s = box.schema.space.create("sync", {is_sync=true}) - --- - ... - - Then, create an index: - - .. code-block:: console - - app:instance001> _ = s:create_index('pk') - --- - ... - - After that, use ``box.ctl.promote()`` function to claim a queue: + box_info_synchro:instance002> os.exit(0) + тип Connection was closed. Probably instance process isn't running anymore - .. code-block:: console + On the **first** instance, try to perform some synchronous transactions. + The transactions would hang, because the :ref:`replication.synchro_quorum ` + option is set to `2`, and the second instance is not available: - app:instance001> box.ctl.promote() + .. code-block:: tarantoolsession - Next, perform data manipulations: - - .. code-block:: console - - app:instance001> require('fiber').new(function() box.space.sync:replace{1} end) - --- - - status: suspended - name: lua - id: 130 - ... - app:instance001> require('fiber').new(function() box.space.sync:replace{1} end) + box_info_synchro:instance001> fiber = require('fiber') --- - - status: suspended - name: lua - id: 131 ... - app:instance001> require('fiber').new(function() box.space.sync:replace{1} end) - --- - - status: suspended - name: lua - id: 132 + box_info_synchro:instance001> for i = 1, 3 do fiber.new(function() box.space.sync:replace{i} end) end + --- end ... - If you call the ``box.info.synchro`` command again, - you will see that now there are 3 transactions waiting in the queue: + Call the ``box.info.synchro`` command on the first instance again: - .. code-block:: console + .. code-block:: tarantoolsession - app:instance001> box.info.synchro + box_info_synchro:instance001> box.info.synchro --- - queue: owner: 1 confirm_lag: 0 term: 2 - age: 0 - len: 0 + age: 5.2658250015229 + len: 3 busy: false quorum: 2 ... + + The ``len`` field is now equal to 3. It means that there are 3 transactions waiting in the queue. From 42ffb40746a2b4274a39e0d8690360cf1c319906 Mon Sep 17 00:00:00 2001 From: Kseniia Antonova Date: Thu, 17 Oct 2024 11:15:39 +0300 Subject: [PATCH 4/5] Fix typos --- doc/reference/reference_lua/box_info/synchro.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/doc/reference/reference_lua/box_info/synchro.rst b/doc/reference/reference_lua/box_info/synchro.rst index 85927b553..e63a38c88 100644 --- a/doc/reference/reference_lua/box_info/synchro.rst +++ b/doc/reference/reference_lua/box_info/synchro.rst @@ -87,10 +87,10 @@ box.info.synchro - ``instance002`` is a follower instance. .. literalinclude:: /code_snippets/snippets/replication/instances.enabled/box_info_synchro/config.yaml - :language: yaml - :start-at: groups - :end-at: 3302 - :dedent: + :language: yaml + :start-at: groups + :end-at: 3302 + :dedent: On the **first** instance, grant the user with the ``super`` role: @@ -149,7 +149,7 @@ box.info.synchro --- ... box_info_synchro:instance001> for i = 1, 3 do fiber.new(function() box.space.sync:replace{i} end) end - --- end + --- ... Call the ``box.info.synchro`` command on the first instance again: From 2a5a077cca7f0f26ae8a49d6472755f51a354ef4 Mon Sep 17 00:00:00 2001 From: Kseniia Antonova <73473519+xuniq@users.noreply.github.com> Date: Fri, 18 Oct 2024 10:38:24 +0300 Subject: [PATCH 5/5] Apply suggestions from code review Co-authored-by: Elena Shebunyaeva --- doc/reference/reference_lua/box_info/synchro.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/reference/reference_lua/box_info/synchro.rst b/doc/reference/reference_lua/box_info/synchro.rst index e63a38c88..9319e38fa 100644 --- a/doc/reference/reference_lua/box_info/synchro.rst +++ b/doc/reference/reference_lua/box_info/synchro.rst @@ -30,7 +30,7 @@ box.info.synchro When Raft election is enabled and :ref:`replication.election_mode ` is set to ``candidate``, the new Raft leader claims the queue automatically after winning the elections. It means that the value of ``box.info.synchro.queue.owner`` becomes equal to :ref:`box.info.election.leader `. - When Raft enabled, no manual intervention with ``box.ctl.promote()`` or ``box.ctl.demote()`` is required. + When Raft is enabled, no manual intervention with ``box.ctl.promote()`` or ``box.ctl.demote()`` is required. - ``term`` (since version :doc:`2.10.0 `) -- current queue term. It contains the term of the last ``PROMOTE`` request. @@ -98,13 +98,13 @@ box.info.synchro box_info_synchro:instance001> box.schema.user.grant('guest', 'super') - After that, use ``box.ctl.promote()`` function to claim the queue: + After that, use the ``box.ctl.promote()`` function to claim the queue: .. code-block:: tarantoolsession box_info_synchro:instance001> box.ctl.promote() - Create a space called ``sync`` and enable synchronous replication on this space: + Create a space named ``sync`` and enable synchronous replication on this space: .. code-block:: tarantoolsession