From ba2a698240a84062ce87b3a06bbbc5908d0a50b4 Mon Sep 17 00:00:00 2001 From: Kseniia Antonova Date: Thu, 5 May 2022 11:20:49 +0300 Subject: [PATCH 01/13] Add new functions to module box --- doc/reference/reference_lua/box.rst | 1 + doc/reference/reference_lua/box_watchers.rst | 59 ++++++++ .../reference_lua/box_watchers/broadcast.rst | 15 ++ .../reference_lua/box_watchers/watch.rst | 15 ++ doc/reference/reference_lua/net_box.rst | 42 +++++- doc/reference/reference_rock/vshard/index.rst | 19 +-- .../reference_rock/vshard/vshard_pub-sub.rst | 135 ++++++++++++++++++ 7 files changed, 275 insertions(+), 11 deletions(-) create mode 100644 doc/reference/reference_lua/box_watchers.rst create mode 100644 doc/reference/reference_lua/box_watchers/broadcast.rst create mode 100644 doc/reference/reference_lua/box_watchers/watch.rst create mode 100644 doc/reference/reference_rock/vshard/vshard_pub-sub.rst diff --git a/doc/reference/reference_lua/box.rst b/doc/reference/reference_lua/box.rst index 1868881ab..218d709ed 100644 --- a/doc/reference/reference_lua/box.rst +++ b/doc/reference/reference_lua/box.rst @@ -34,6 +34,7 @@ with ``box``, with no arguments. The ``box`` module contains: box_space box_stat box_tuple + box_watchers box_txn_management box_sql diff --git a/doc/reference/reference_lua/box_watchers.rst b/doc/reference/reference_lua/box_watchers.rst new file mode 100644 index 000000000..242a88293 --- /dev/null +++ b/doc/reference/reference_lua/box_watchers.rst @@ -0,0 +1,59 @@ +.. _box-watchers: + +-------------------------------------------------------------------------------- +Functions for watchers +-------------------------------------------------------------------------------- + +The ``box`` module contains some functions related to the event subscriptions, also known as watchers. +The subscriptions are used to inform a client about server-side events. +To see the list of the built-in events in Tarantool, +check the :doc:`pub/sub ` section. + +.. glossary:: + + Watcher + A watcher is a :doc:`callback ` invoked on a state change. + To create a local watcher, use the ``box.watch()`` function. + + State + A state is a key-value pair stored internally. + The key is a string. + The value is any type that could be encoded as MsgPack. + To update a state, use the ``box.broadcast()`` function. + +A watcher callback is passed the key for which it was registered and the current key data. +A watcher callback is always invoked in a new fiber, so it's okay to yield in it. +A watcher callback is never executed in parallel with itself: +if the key updates while the callback is running, +the callback will be invoked with the new value as soon as it returns. + +The watch() function returns a watcher handle, which can be used to unregister the watcher +if it is not needed anymore by calling ``w:unregister()``. + +Below is a list of all functions and members. + +.. container:: table + + .. rst-class:: left-align-column-1 + .. rst-class:: left-align-column-2 + + .. list-table:: + :widths: 25 75 + :header-rows: 1 + + * - Name + - Use + + * - :doc:`./box_watchers/watch` + - Create a local watcher. + To create a remote watcher, see the :ref:`conn:watch() ` function + in the ``net.box`` module. + + * - :doc:`./box_watchers/broadcast` + - Update a state. + +.. toctree:: + :hidden: + + box_watchers/watch + box_watchers/broadcast \ No newline at end of file diff --git a/doc/reference/reference_lua/box_watchers/broadcast.rst b/doc/reference/reference_lua/box_watchers/broadcast.rst new file mode 100644 index 000000000..fb117e457 --- /dev/null +++ b/doc/reference/reference_lua/box_watchers/broadcast.rst @@ -0,0 +1,15 @@ +.. _box-broadcast: + +================================================================================ +box.broadcast() +================================================================================ + +.. function:: box.broadcast(key, value) + + //Description + + :param string key: a key name to subscribe to + :param ?mgspack value: + + :return: + :rtype: ? \ No newline at end of file diff --git a/doc/reference/reference_lua/box_watchers/watch.rst b/doc/reference/reference_lua/box_watchers/watch.rst new file mode 100644 index 000000000..4165cbab7 --- /dev/null +++ b/doc/reference/reference_lua/box_watchers/watch.rst @@ -0,0 +1,15 @@ +.. _box-watch: + +================================================================================ +box.watch() +================================================================================ + +.. function:: box.watch(key, func) + + Subscribe to events broadcast by a remote host. + + :param string key: a key name to subscribe to + :param function func: a callback to invoke when the key value is updated + + :return: a watcher handle that can be used to unregister the watcher + :rtype: ? \ No newline at end of file diff --git a/doc/reference/reference_lua/net_box.rst b/doc/reference/reference_lua/net_box.rst index c154c0a7e..529ed3562 100644 --- a/doc/reference/reference_lua/net_box.rst +++ b/doc/reference/reference_lua/net_box.rst @@ -125,14 +125,16 @@ Below is a list of all ``net.box`` functions. * - :ref:`conn:call() ` - Call a stored procedure * - :ref:`conn:timeout() ` - - Set a timeout + - Set a timeout + * - :ref:`conn:watch() ` + - Subscribe to events broadcast by a remote host * - :ref:`conn:on_connect() ` - Define a connect trigger * - :ref:`conn:on_disconnect() ` - Define a disconnect trigger * - :ref:`conn:on_schema_reload() ` - Define a trigger when schema is modified - * - :ref:`conn:new_stream() ` + * - :ref:`conn:new_stream() ` - Create a stream * - :ref:`stream:begin() ` - Begin a stream transaction @@ -533,7 +535,43 @@ Below is a list of all ``net.box`` functions. - B ... + .. _conn-watch: + .. method:: watch(key, func) + + Subscribe to events broadcast by a remote host. + + :param string key: a key name to subscribe to + :param function func: a callback to invoke when the key value is updated + :return: a watcher handle that can be used to unregister the watcher + :rtype: ? + + .. admonition:: note + + Garbage collection of a watcher handle doesn't result in unregistering the watcher. + It is okay to discard the result of box.watch if the watcher is never going to be unregistered. + + **Example:** + + Server: + + .. code-block:: lua + + -- Broadcast value 123 for key 'foo'. + box.broadcast('foo', 123) + + Client: + + .. code-block:: lua + + conn = net.box.connect(URI) + -- Subscribe to updates of key 'foo'. + w = conn:watch('foo', function(key, value) + assert(key == 'foo') + -- do something with value + end) + -- Unregister the watcher when it's no longer needed. + w:unregister() .. _conn-timeout: diff --git a/doc/reference/reference_rock/vshard/index.rst b/doc/reference/reference_rock/vshard/index.rst index 41ee7813b..c86b16216 100644 --- a/doc/reference/reference_rock/vshard/index.rst +++ b/doc/reference/reference_rock/vshard/index.rst @@ -11,13 +11,14 @@ scaling in Tarantool. Check out the :ref:`Quick start guide ` -- or dive into the complete ``vshard`` documentation: -.. toctree:: - :maxdepth: 2 - :numbered: 0 +.. toctree:: + :maxdepth: 2 + :numbered: 0 - vshard_summary_index - vshard_architecture - vshard_admin - vshard_quick - vshard_ref - vshard_api + vshard_summary_index + vshard_architecture + vshard_admin + vshard_quick + vshard_ref + vshard_api + vshard_pub-sub diff --git a/doc/reference/reference_rock/vshard/vshard_pub-sub.rst b/doc/reference/reference_rock/vshard/vshard_pub-sub.rst new file mode 100644 index 000000000..88250aa25 --- /dev/null +++ b/doc/reference/reference_rock/vshard/vshard_pub-sub.rst @@ -0,0 +1,135 @@ +.. _vshard-pubsub: + +Pub/sub system and its events +============================= + +Pub/sub is a notification system for Tarantool events. +It is related to one-time subscriptions. + +Events that the system will process: + +box.id +box.status +box.election +box.schema + +As a reaction to each event, the server sends back specific IPROTO fields. + +Built-in events for pub/sub +--------------------------- + +The important purpose of the built-in events is to learn who is the +master, unless it is defined in an application specific way. Knowing who +is the master is necessary to send changes to a correct instance, and +probably make reads of the most actual data if it is important. Also +defined more built-in events for other mutable properties like leader +state change, his election role and election term, schema version change +and instance state. + +Built-in events have a special naming schema --ё their name always starts +with box.. The prefix is reserved for built-in events. Creating new events +with this prefix is banned. Below is a list of all the events + their names +and values: + +.. container:: table + + .. list-table:: + :widths: 20 40 40 + + * - Built-in event + - Description + - Value + + * - box.id + - Identification of the instance. Changes are extra rare. Some + values never change or change only once. For example, instance UUID never + changes after the first box.cfg. But is not known before box.cfg is called. + Replicaset UUID is unknown until the instance joins to a replicaset or + bootsa new one, but the events are supposed to start working before that - + right at listen launch. Instance numeric ID is known only after + registration. On anonymous replicas is 0 until they are registered officially. + - .. code-block:: lua + + { + MP_STR “id”: MP_UINT; box.info.id, + MP_STR “instance_uuid”: MP_UUID; box.info.uuid, + MP_STR “replicaset_uuid”: MP_UUID box.info.cluster.uuid, + } + + * - box.status + - Generic blob about instance status. It is most commonly used + and not frequently changed config options and box.info fields.] + - .. code-block:: lua + + { + MP_STR “is_ro”: MP_BOOL box.info.ro, + MP_STR “is_ro_cfg”: MP_BOOL box.cfg.read_only, + MP_STR “status”: MP_STR box.info.status, + } + + * - box.election + - All the needed parts of box.info.election needed to find who is the most recent writable leader. + - .. code-block:: lua + + { + MP_STR “term”: MP_UINT box.info.election.term, + MP_STR “role”: MP_STR box.info.election.state, + MP_STR “is_ro”: MP_BOOL box.info.ro, + MP_STR “leader”: MP_UINT box.info.election.leader, + } + + * - box.schema + - Schema-related data. Currently it is only version. + - .. code-block:: lua + + { + MP_STR “version”: MP_UINT schema_version, + } + +Built-in events can't be override. Meaning, users can't be able to call +box.broadcast(‘box.id’, any_data) etc. + +The events are available from the very beginning as not MP_NIL. It's +necessary for supported local subscriptions. Otherwise, there is no way to detect +whether an event is even supported at all by this Tarantool version. If +events are broadcast before box.cfg{}, then the following values will +available: +box.id = {} +box.schema = {} +box.status = {} +box.election = {} + +This way, the users will be able to distinguish an event being not supported +at all from ``box.cfg{}`` being not called yet. Otherwise they would need to +parse ``_TARANTOOL`` version string locally and peer_version in net.box. + +Usage example +------------- + +.. code-block:: lua + + conn = net.box.connect(URI) + -- Subscribe to updates of key 'box.id' + w = conn:watch('box.id', function(key, value) + assert(key == 'box.id') + -- do something with value + end) + -- or to updates of key 'box.status' + w = conn:watch('box.status', function(key, value) + assert(key == 'box.status') + -- do something with value + end) + -- or to updates of key 'box.election' + w = conn:watch('box.election', function(key, value) + assert(key == 'box.election') + -- do something with value + end) + -- or to updates of key 'box.schema' + w = conn:watch('box.schema', function(key, value) + assert(key == 'box.schema') + -- do something with value + end) + -- Unregister the watcher when it's no longer needed. + w:unregister() + + From 7c49745ee42cd1d418e07f555b7c9d3a61833a4d Mon Sep 17 00:00:00 2001 From: Kseniia Antonova Date: Thu, 23 Jun 2022 11:37:02 +0300 Subject: [PATCH 02/13] Update watch and broadcast methods --- doc/reference/reference_lua/box_watchers.rst | 6 +- .../reference_lua/box_watchers/broadcast.rst | 19 ++++-- .../reference_lua/box_watchers/watch.rst | 19 +++++- doc/reference/reference_lua/net_box.rst | 31 +++++++-- .../reference_rock/vshard/vshard_pub-sub.rst | 65 ++++++++++--------- 5 files changed, 93 insertions(+), 47 deletions(-) diff --git a/doc/reference/reference_lua/box_watchers.rst b/doc/reference/reference_lua/box_watchers.rst index 242a88293..eae226d66 100644 --- a/doc/reference/reference_lua/box_watchers.rst +++ b/doc/reference/reference_lua/box_watchers.rst @@ -27,10 +27,10 @@ A watcher callback is never executed in parallel with itself: if the key updates while the callback is running, the callback will be invoked with the new value as soon as it returns. -The watch() function returns a watcher handle, which can be used to unregister the watcher -if it is not needed anymore by calling ``w:unregister()``. +The ``watch()`` function returns a watcher handle, which can be used to unregister the watcher +if it is not needed anymore. -Below is a list of all functions and members. +Below is a list of all functions related to watchers. .. container:: table diff --git a/doc/reference/reference_lua/box_watchers/broadcast.rst b/doc/reference/reference_lua/box_watchers/broadcast.rst index fb117e457..edd7c7e60 100644 --- a/doc/reference/reference_lua/box_watchers/broadcast.rst +++ b/doc/reference/reference_lua/box_watchers/broadcast.rst @@ -4,12 +4,21 @@ box.broadcast() ================================================================================ -.. function:: box.broadcast(key, value) +.. function:: box.broadcast(key, value) - //Description + Update the value of a given key and inform all the key watchers about the update. - :param string key: a key name to subscribe to - :param ?mgspack value: + :param string key: a key name of an event to subscribe to + :param value: any data that can be encoded in MsgPack :return: - :rtype: ? \ No newline at end of file + :rtype: ? + + **Example:** + + .. code-block:: lua + + -- Broadcast value 123 for the key 'foo'. + box.broadcast('foo', 123) + + diff --git a/doc/reference/reference_lua/box_watchers/watch.rst b/doc/reference/reference_lua/box_watchers/watch.rst index 4165cbab7..72d1934fa 100644 --- a/doc/reference/reference_lua/box_watchers/watch.rst +++ b/doc/reference/reference_lua/box_watchers/watch.rst @@ -8,8 +8,23 @@ box.watch() Subscribe to events broadcast by a remote host. - :param string key: a key name to subscribe to + :param string key: a key name of an event to subscribe to :param function func: a callback to invoke when the key value is updated :return: a watcher handle that can be used to unregister the watcher - :rtype: ? \ No newline at end of file + :rtype: ? + + **Example:** + + .. code-block:: lua + + -- Broadcast value 123 for key 'foo'. + box.broadcast('foo', 123) + -- Subscribe to updates of key 'foo'. + w = box.watch('foo', function(key, value) + assert(key == 'foo') + -- do something with value + end) + -- Unregister the watcher when it's no longer needed. + w:unregister() + diff --git a/doc/reference/reference_lua/net_box.rst b/doc/reference/reference_lua/net_box.rst index 529ed3562..2fec5573c 100644 --- a/doc/reference/reference_lua/net_box.rst +++ b/doc/reference/reference_lua/net_box.rst @@ -541,15 +541,34 @@ Below is a list of all ``net.box`` functions. Subscribe to events broadcast by a remote host. - :param string key: a key name to subscribe to + :param string key: a key name of an event to subscribe to :param function func: a callback to invoke when the key value is updated :return: a watcher handle that can be used to unregister the watcher :rtype: ? - .. admonition:: note + Keep in mind that garbage collection of a watcher handle doesn't result in unregistering the watcher. + It is okay to discard the result of ``box.watch`` if the watcher is never going to be unregistered. - Garbage collection of a watcher handle doesn't result in unregistering the watcher. - It is okay to discard the result of box.watch if the watcher is never going to be unregistered. + Using the new watch method of a net.box connection, one can subscribe + to events broadcast by a remote host. + The method has the same syntax as the :doc:`box.watch() ` + function, which is used for subscribing to events locally. + + A watcher callback is first invoked unconditionally after the watcher registration. + Subsequent invocations are triggered by :doc:`box.broadcast() ` + called on the remote host. + A watcher callback is passed the name of the key the watcher was subscribed to and the current key value. + A watcher callback is always executed in a new fiber so it's okay to yield inside it. + A watcher callback never runs in parallel with itself: if the key to which a watcher is subscribed + is updated while the watcher callback is running, the callback will be invoked again with the new value as + soon as it returns. + + Watchers survive reconnect (see ``reconnect_after`` connection option): + all registered watchers are automatically resubscribed as soon as the + connection is reestablished. + + If a remote host supports watchers, the 'watchers' key will be set in + connection's ``peer_protocol_features``. **Example:** @@ -557,7 +576,7 @@ Below is a list of all ``net.box`` functions. .. code-block:: lua - -- Broadcast value 123 for key 'foo'. + -- Broadcast value 123 for the key 'foo'. box.broadcast('foo', 123) Client: @@ -570,7 +589,7 @@ Below is a list of all ``net.box`` functions. assert(key == 'foo') -- do something with value end) - -- Unregister the watcher when it's no longer needed. + -- Unregister the watcher if it is no longer needed. w:unregister() .. _conn-timeout: diff --git a/doc/reference/reference_rock/vshard/vshard_pub-sub.rst b/doc/reference/reference_rock/vshard/vshard_pub-sub.rst index 88250aa25..8901c1285 100644 --- a/doc/reference/reference_rock/vshard/vshard_pub-sub.rst +++ b/doc/reference/reference_rock/vshard/vshard_pub-sub.rst @@ -6,30 +6,33 @@ Pub/sub system and its events Pub/sub is a notification system for Tarantool events. It is related to one-time subscriptions. -Events that the system will process: +The system processes the following events: -box.id -box.status -box.election -box.schema +* ``box.id`` +* ``box.status`` +* ``box.election`` +* ``box.schema`` As a reaction to each event, the server sends back specific IPROTO fields. Built-in events for pub/sub --------------------------- -The important purpose of the built-in events is to learn who is the -master, unless it is defined in an application specific way. Knowing who -is the master is necessary to send changes to a correct instance, and -probably make reads of the most actual data if it is important. Also -defined more built-in events for other mutable properties like leader +The important purpose of the built-in events is master-discovering. +It is necessary to know the master node in order to send changes to a correct instance, +or read the most actual data. +Also, defined more built-in events for other mutable properties like leader state change, his election role and election term, schema version change and instance state. -Built-in events have a special naming schema --ё their name always starts -with box.. The prefix is reserved for built-in events. Creating new events -with this prefix is banned. Below is a list of all the events + their names -and values: +Built-in events have a special naming schema -- theirs name always starts with the ``box`` prefix. +This prefix is reserved for the built-in events. It means that you can't create new events with it. + +Keep in mind that built-in events can't be overridden. +It means that the user can't call +:doc:`box.broadcast('box.id', any_data) `. + +Below is a list of all the events. It includes the name, description, and value: .. container:: table @@ -45,7 +48,7 @@ and values: values never change or change only once. For example, instance UUID never changes after the first box.cfg. But is not known before box.cfg is called. Replicaset UUID is unknown until the instance joins to a replicaset or - bootsa new one, but the events are supposed to start working before that - + boots a new one, but the events are supposed to start working before that -- right at listen launch. Instance numeric ID is known only after registration. On anonymous replicas is 0 until they are registered officially. - .. code-block:: lua @@ -86,22 +89,22 @@ and values: MP_STR “version”: MP_UINT schema_version, } -Built-in events can't be override. Meaning, users can't be able to call -box.broadcast(‘box.id’, any_data) etc. - -The events are available from the very beginning as not MP_NIL. It's -necessary for supported local subscriptions. Otherwise, there is no way to detect -whether an event is even supported at all by this Tarantool version. If -events are broadcast before box.cfg{}, then the following values will -available: -box.id = {} -box.schema = {} -box.status = {} -box.election = {} - -This way, the users will be able to distinguish an event being not supported -at all from ``box.cfg{}`` being not called yet. Otherwise they would need to -parse ``_TARANTOOL`` version string locally and peer_version in net.box. +The events are available from the very beginning as not MP_NIL. +It is necessary for supported local subscriptions. +Otherwise, there is no way to detect whether an event is even supported at all by this Tarantool version. +If the events are broadcast before :doc:`box.cfg{} `, +then the following values will be available: + +.. code-block:: lua + + box.id = {} + box.schema = {} + box.status = {} + box.election = {} + +This way, the users can distinguish an event being not supported +at all from ``box.cfg{}`` being not called yet. +Otherwise, they would need to parse ``_TARANTOOL`` version string locally and ``peer_version`` in ``net.box``. Usage example ------------- From d14c852c478d85064d4f8726179118bb899fea8e Mon Sep 17 00:00:00 2001 From: Kseniia Antonova Date: Fri, 24 Jun 2022 16:15:41 +0300 Subject: [PATCH 03/13] Update pub/sub --- .../reference_lua/box_watchers/broadcast.rst | 3 +- .../reference_lua/box_watchers/watch.rst | 3 +- doc/reference/reference_lua/net_box.rst | 1 - .../reference_rock/vshard/vshard_index.rst | 17 ++-- .../reference_rock/vshard/vshard_pub-sub.rst | 77 ++++++++++--------- 5 files changed, 51 insertions(+), 50 deletions(-) diff --git a/doc/reference/reference_lua/box_watchers/broadcast.rst b/doc/reference/reference_lua/box_watchers/broadcast.rst index edd7c7e60..2ae65ece6 100644 --- a/doc/reference/reference_lua/box_watchers/broadcast.rst +++ b/doc/reference/reference_lua/box_watchers/broadcast.rst @@ -11,8 +11,7 @@ box.broadcast() :param string key: a key name of an event to subscribe to :param value: any data that can be encoded in MsgPack - :return: - :rtype: ? + :return: none **Example:** diff --git a/doc/reference/reference_lua/box_watchers/watch.rst b/doc/reference/reference_lua/box_watchers/watch.rst index 72d1934fa..6e0cef519 100644 --- a/doc/reference/reference_lua/box_watchers/watch.rst +++ b/doc/reference/reference_lua/box_watchers/watch.rst @@ -6,13 +6,12 @@ box.watch() .. function:: box.watch(key, func) - Subscribe to events broadcast by a remote host. + Subscribe to events broadcast by a local host. :param string key: a key name of an event to subscribe to :param function func: a callback to invoke when the key value is updated :return: a watcher handle that can be used to unregister the watcher - :rtype: ? **Example:** diff --git a/doc/reference/reference_lua/net_box.rst b/doc/reference/reference_lua/net_box.rst index 2fec5573c..9f356b33c 100644 --- a/doc/reference/reference_lua/net_box.rst +++ b/doc/reference/reference_lua/net_box.rst @@ -544,7 +544,6 @@ Below is a list of all ``net.box`` functions. :param string key: a key name of an event to subscribe to :param function func: a callback to invoke when the key value is updated :return: a watcher handle that can be used to unregister the watcher - :rtype: ? Keep in mind that garbage collection of a watcher handle doesn't result in unregistering the watcher. It is okay to discard the result of ``box.watch`` if the watcher is never going to be unregistered. diff --git a/doc/reference/reference_rock/vshard/vshard_index.rst b/doc/reference/reference_rock/vshard/vshard_index.rst index 053097866..2f34faafa 100644 --- a/doc/reference/reference_rock/vshard/vshard_index.rst +++ b/doc/reference/reference_rock/vshard/vshard_index.rst @@ -11,12 +11,13 @@ Sharding Check out the :ref:`Quick start guide ` -- or dive into the complete ``vshard`` documentation: -.. toctree:: - :maxdepth: 2 - :numbered: 0 +.. toctree:: + :maxdepth: 2 + :numbered: 0 + + vshard_architecture + vshard_admin + vshard_quick + vshard_ref + vshard_api - vshard_architecture - vshard_admin - vshard_quick - vshard_ref - vshard_api diff --git a/doc/reference/reference_rock/vshard/vshard_pub-sub.rst b/doc/reference/reference_rock/vshard/vshard_pub-sub.rst index 8901c1285..eb255d4a9 100644 --- a/doc/reference/reference_rock/vshard/vshard_pub-sub.rst +++ b/doc/reference/reference_rock/vshard/vshard_pub-sub.rst @@ -18,11 +18,11 @@ As a reaction to each event, the server sends back specific IPROTO fields. Built-in events for pub/sub --------------------------- -The important purpose of the built-in events is master-discovering. +First, the important purpose of the built-in events is :ref:`master discovery `. It is necessary to know the master node in order to send changes to a correct instance, or read the most actual data. -Also, defined more built-in events for other mutable properties like leader -state change, his election role and election term, schema version change +Also, there are more built-in events for other mutable properties, like leader +state change, his election role and term, schema version change, and instance state. Built-in events have a special naming schema -- theirs name always starts with the ``box`` prefix. @@ -32,16 +32,15 @@ Keep in mind that built-in events can't be overridden. It means that the user can't call :doc:`box.broadcast('box.id', any_data) `. -Below is a list of all the events. It includes the name, description, and value: +Below is a table listing all the events, which includes the event name and its description. .. container:: table .. list-table:: - :widths: 20 40 40 + :widths: 50 50 * - Built-in event - Description - - Value * - box.id - Identification of the instance. Changes are extra rare. Some @@ -51,45 +50,49 @@ Below is a list of all the events. It includes the name, description, and value: boots a new one, but the events are supposed to start working before that -- right at listen launch. Instance numeric ID is known only after registration. On anonymous replicas is 0 until they are registered officially. - - .. code-block:: lua - - { - MP_STR “id”: MP_UINT; box.info.id, - MP_STR “instance_uuid”: MP_UUID; box.info.uuid, - MP_STR “replicaset_uuid”: MP_UUID box.info.cluster.uuid, - } * - box.status - Generic blob about instance status. It is most commonly used - and not frequently changed config options and box.info fields.] - - .. code-block:: lua - - { - MP_STR “is_ro”: MP_BOOL box.info.ro, - MP_STR “is_ro_cfg”: MP_BOOL box.cfg.read_only, - MP_STR “status”: MP_STR box.info.status, - } + and not frequently changed config options and box.info fields. * - box.election - All the needed parts of box.info.election needed to find who is the most recent writable leader. - - .. code-block:: lua - - { - MP_STR “term”: MP_UINT box.info.election.term, - MP_STR “role”: MP_STR box.info.election.state, - MP_STR “is_ro”: MP_BOOL box.info.ro, - MP_STR “leader”: MP_UINT box.info.election.leader, - } * - box.schema - - Schema-related data. Currently it is only version. - - .. code-block:: lua - - { - MP_STR “version”: MP_UINT schema_version, - } - -The events are available from the very beginning as not MP_NIL. + - Schema-related data. Currently, it contains only version. + +The value for each of the built-in events is written in the following code-block:. + +- .. code-block:: lua + + -- box.id value + { + MP_STR “id”: MP_UINT; box.info.id, + MP_STR “instance_uuid”: MP_UUID; box.info.uuid, + MP_STR “replicaset_uuid”: MP_UUID box.info.cluster.uuid, + } + + -- box.status value + { + MP_STR “is_ro”: MP_BOOL box.info.ro, + MP_STR “is_ro_cfg”: MP_BOOL box.cfg.read_only, + MP_STR “status”: MP_STR box.info.status, + } + + -- box.election value + { + MP_STR “term”: MP_UINT box.info.election.term, + MP_STR “role”: MP_STR box.info.election.state, + MP_STR “is_ro”: MP_BOOL box.info.ro, + MP_STR “leader”: MP_UINT box.info.election.leader, + } + + -- box.schema value + { + MP_STR “version”: MP_UINT schema_version, + } + +The events are available from the very beginning as not ``MP_NIL``. It is necessary for supported local subscriptions. Otherwise, there is no way to detect whether an event is even supported at all by this Tarantool version. If the events are broadcast before :doc:`box.cfg{} `, From 8ac6786af988e1f8ddb907bc86dd5671cbfebd5b Mon Sep 17 00:00:00 2001 From: Kseniia Antonova Date: Mon, 27 Jun 2022 16:39:03 +0300 Subject: [PATCH 04/13] Proofread text and add links --- doc/reference/reference_lua/box_watchers.rst | 2 +- .../reference_rock/vshard/vshard_pub-sub.rst | 121 +++++++++--------- 2 files changed, 63 insertions(+), 60 deletions(-) diff --git a/doc/reference/reference_lua/box_watchers.rst b/doc/reference/reference_lua/box_watchers.rst index eae226d66..c0c6010a3 100644 --- a/doc/reference/reference_lua/box_watchers.rst +++ b/doc/reference/reference_lua/box_watchers.rst @@ -4,7 +4,7 @@ Functions for watchers -------------------------------------------------------------------------------- -The ``box`` module contains some functions related to the event subscriptions, also known as watchers. +The ``box`` module contains some functions related to event subscriptions, also known as watchers. The subscriptions are used to inform a client about server-side events. To see the list of the built-in events in Tarantool, check the :doc:`pub/sub ` section. diff --git a/doc/reference/reference_rock/vshard/vshard_pub-sub.rst b/doc/reference/reference_rock/vshard/vshard_pub-sub.rst index eb255d4a9..d07cb9428 100644 --- a/doc/reference/reference_rock/vshard/vshard_pub-sub.rst +++ b/doc/reference/reference_rock/vshard/vshard_pub-sub.rst @@ -4,7 +4,7 @@ Pub/sub system and its events ============================= Pub/sub is a notification system for Tarantool events. -It is related to one-time subscriptions. +It is associated with one-time subscriptions. The system processes the following events: @@ -13,26 +13,26 @@ The system processes the following events: * ``box.election`` * ``box.schema`` -As a reaction to each event, the server sends back specific IPROTO fields. +In response to each event, the server sends back certain IPROTO fields. Built-in events for pub/sub --------------------------- -First, the important purpose of the built-in events is :ref:`master discovery `. -It is necessary to know the master node in order to send changes to a correct instance, -or read the most actual data. -Also, there are more built-in events for other mutable properties, like leader -state change, his election role and term, schema version change, -and instance state. +One of the most important purposes of the built-in events is :ref:`master discovery `. +It is necessary to know the master node in order to send changes to a correct instance +or to read the most recent data. +In addition, there are more built-in events for other mutable properties such as the change of the leader's +state, its election role and term, the change of the schema version, +and the state of the instance. -Built-in events have a special naming schema -- theirs name always starts with the ``box`` prefix. -This prefix is reserved for the built-in events. It means that you can't create new events with it. +Built-in events have a special naming schema -- theirs names always start with the ``box`` prefix. +This prefix is reserved for the built-in events. It means that you cannot create new events with it. -Keep in mind that built-in events can't be overridden. -It means that the user can't call +Note that built-in events can't be overridden. +It means that the user cannot call :doc:`box.broadcast('box.id', any_data) `. -Below is a table listing all the events, which includes the event name and its description. +Below is a table of all events that includes the name of the event and its description. .. container:: table @@ -43,60 +43,63 @@ Below is a table listing all the events, which includes the event name and its d - Description * - box.id - - Identification of the instance. Changes are extra rare. Some - values never change or change only once. For example, instance UUID never - changes after the first box.cfg. But is not known before box.cfg is called. - Replicaset UUID is unknown until the instance joins to a replicaset or - boots a new one, but the events are supposed to start working before that -- - right at listen launch. Instance numeric ID is known only after - registration. On anonymous replicas is 0 until they are registered officially. + - Identification of the instance. Changes are particelarly rare. Some + values never change or change only once. For example, the UUID of the instance never + changes after the first :doc:`box.cfg `. + However, it is not known before ``box.cfg`` is called. + The replicaset UUID is unknown until the instance joins a replicaset or + boots a new one, but events are supposed to start working before then -- + right when the listen launches. The numeric instance ID is known only after + registration. For anonymous replicas, the value is ``0`` until they are officially registered. * - box.status - - Generic blob about instance status. It is most commonly used - and not frequently changed config options and box.info fields. + - Generic blob about the instance status. There are the most frequently used + and not frequently changed config options and :doc:`box.info ` + fields. * - box.election - - All the needed parts of box.info.election needed to find who is the most recent writable leader. + - All the required parts of :doc:`box.info.election ` + needed to find out who is the most recent writable leader. * - box.schema - - Schema-related data. Currently, it contains only version. + - Schema-related data. Currently, it contains only the version. The value for each of the built-in events is written in the following code-block:. -- .. code-block:: lua - - -- box.id value - { - MP_STR “id”: MP_UINT; box.info.id, - MP_STR “instance_uuid”: MP_UUID; box.info.uuid, - MP_STR “replicaset_uuid”: MP_UUID box.info.cluster.uuid, - } - - -- box.status value - { - MP_STR “is_ro”: MP_BOOL box.info.ro, - MP_STR “is_ro_cfg”: MP_BOOL box.cfg.read_only, - MP_STR “status”: MP_STR box.info.status, - } - - -- box.election value - { - MP_STR “term”: MP_UINT box.info.election.term, - MP_STR “role”: MP_STR box.info.election.state, - MP_STR “is_ro”: MP_BOOL box.info.ro, - MP_STR “leader”: MP_UINT box.info.election.leader, - } - - -- box.schema value - { - MP_STR “version”: MP_UINT schema_version, - } - -The events are available from the very beginning as not ``MP_NIL``. +.. code-block:: lua + + -- box.id value + { + MP_STR “id”: MP_UINT; box.info.id, + MP_STR “instance_uuid”: MP_UUID; box.info.uuid, + MP_STR “replicaset_uuid”: MP_UUID box.info.cluster.uuid, + } + + -- box.status value + { + MP_STR “is_ro”: MP_BOOL box.info.ro, + MP_STR “is_ro_cfg”: MP_BOOL box.cfg.read_only, + MP_STR “status”: MP_STR box.info.status, + } + + -- box.election value + { + MP_STR “term”: MP_UINT box.info.election.term, + MP_STR “role”: MP_STR box.info.election.state, + MP_STR “is_ro”: MP_BOOL box.info.ro, + MP_STR “leader”: MP_UINT box.info.election.leader, + } + + -- box.schema value + { + MP_STR “version”: MP_UINT schema_version, + } + +The events are available from the beginning as non-``MP_NIL``. It is necessary for supported local subscriptions. -Otherwise, there is no way to detect whether an event is even supported at all by this Tarantool version. +Otherwise, there is no way to detect whether an event is supported at all by this Tarantool version. If the events are broadcast before :doc:`box.cfg{} `, -then the following values will be available: +then the following values are available: .. code-block:: lua @@ -105,9 +108,9 @@ then the following values will be available: box.status = {} box.election = {} -This way, the users can distinguish an event being not supported -at all from ``box.cfg{}`` being not called yet. -Otherwise, they would need to parse ``_TARANTOOL`` version string locally and ``peer_version`` in ``net.box``. +This way, users can distinguish if an event being not supported +at all or if ``box.cfg{}`` has not beeen called yet. +Otherwise, they would need to parse the ``_TARANTOOL`` version string locally and the ``peer_version`` in ``net.box``. Usage example ------------- From cd26e3166255b0bb1b006ce5de3c91d6523aa0e1 Mon Sep 17 00:00:00 2001 From: Kseniia Antonova Date: Mon, 27 Jun 2022 17:25:58 +0300 Subject: [PATCH 05/13] Proofread text, update links --- doc/reference/reference_lua/box_watchers.rst | 18 ++++++------- .../reference_lua/box_watchers/broadcast.rst | 4 +-- .../reference_lua/box_watchers/watch.rst | 6 ++--- doc/reference/reference_lua/net_box.rst | 25 +++++++++---------- .../reference_rock/vshard/vshard_pub-sub.rst | 10 ++++---- 5 files changed, 31 insertions(+), 32 deletions(-) diff --git a/doc/reference/reference_lua/box_watchers.rst b/doc/reference/reference_lua/box_watchers.rst index c0c6010a3..5c8f3cf62 100644 --- a/doc/reference/reference_lua/box_watchers.rst +++ b/doc/reference/reference_lua/box_watchers.rst @@ -7,28 +7,28 @@ Functions for watchers The ``box`` module contains some functions related to event subscriptions, also known as watchers. The subscriptions are used to inform a client about server-side events. To see the list of the built-in events in Tarantool, -check the :doc:`pub/sub ` section. +check the :doc:`pub/sub ` section. .. glossary:: Watcher - A watcher is a :doc:`callback ` invoked on a state change. + A watcher is a :doc:`callback ` that is invoked when a state change occurs. To create a local watcher, use the ``box.watch()`` function. State - A state is a key-value pair stored internally. + A state is an internally stored key-value pair. The key is a string. - The value is any type that could be encoded as MsgPack. + The value is an arbitrary type that can be encoded as MsgPack. To update a state, use the ``box.broadcast()`` function. A watcher callback is passed the key for which it was registered and the current key data. -A watcher callback is always invoked in a new fiber, so it's okay to yield in it. -A watcher callback is never executed in parallel with itself: -if the key updates while the callback is running, +A watcher callback is always called in a new fiber, so it is okay to yield in it. +A watcher callback is never executed in parallel with itself. +If the key is updated while the callback is running, the callback will be invoked with the new value as soon as it returns. -The ``watch()`` function returns a watcher handle, which can be used to unregister the watcher -if it is not needed anymore. +The ``watch()`` function returns a watcher handle that can be used to unregister the watcher +when it is no longer needed. Below is a list of all functions related to watchers. diff --git a/doc/reference/reference_lua/box_watchers/broadcast.rst b/doc/reference/reference_lua/box_watchers/broadcast.rst index 2ae65ece6..ac2dfa1ef 100644 --- a/doc/reference/reference_lua/box_watchers/broadcast.rst +++ b/doc/reference/reference_lua/box_watchers/broadcast.rst @@ -6,7 +6,7 @@ box.broadcast() .. function:: box.broadcast(key, value) - Update the value of a given key and inform all the key watchers about the update. + Update the value of a particular key and notify all key watchers of the update. :param string key: a key name of an event to subscribe to :param value: any data that can be encoded in MsgPack @@ -17,7 +17,7 @@ box.broadcast() .. code-block:: lua - -- Broadcast value 123 for the key 'foo'. + -- Broadcast value 123 for the 'foo' key. box.broadcast('foo', 123) diff --git a/doc/reference/reference_lua/box_watchers/watch.rst b/doc/reference/reference_lua/box_watchers/watch.rst index 6e0cef519..ff015b9d5 100644 --- a/doc/reference/reference_lua/box_watchers/watch.rst +++ b/doc/reference/reference_lua/box_watchers/watch.rst @@ -17,13 +17,13 @@ box.watch() .. code-block:: lua - -- Broadcast value 123 for key 'foo'. + -- Broadcast value 123 for the 'foo' key. box.broadcast('foo', 123) - -- Subscribe to updates of key 'foo'. + -- Subscribe to updates of the 'foo' key. w = box.watch('foo', function(key, value) assert(key == 'foo') -- do something with value end) - -- Unregister the watcher when it's no longer needed. + -- Unregister the watcher when it is no longer needed. w:unregister() diff --git a/doc/reference/reference_lua/net_box.rst b/doc/reference/reference_lua/net_box.rst index 9f356b33c..2840eb3ef 100644 --- a/doc/reference/reference_lua/net_box.rst +++ b/doc/reference/reference_lua/net_box.rst @@ -545,30 +545,29 @@ Below is a list of all ``net.box`` functions. :param function func: a callback to invoke when the key value is updated :return: a watcher handle that can be used to unregister the watcher - Keep in mind that garbage collection of a watcher handle doesn't result in unregistering the watcher. - It is okay to discard the result of ``box.watch`` if the watcher is never going to be unregistered. - - Using the new watch method of a net.box connection, one can subscribe - to events broadcast by a remote host. - The method has the same syntax as the :doc:`box.watch() ` + Using the ``watch`` method of a net.box connection, you can subscribe to events broadcast by a remote host. + The method has the same syntax as the :doc:`box.watch() ` function, which is used for subscribing to events locally. A watcher callback is first invoked unconditionally after the watcher registration. Subsequent invocations are triggered by :doc:`box.broadcast() ` called on the remote host. A watcher callback is passed the name of the key the watcher was subscribed to and the current key value. - A watcher callback is always executed in a new fiber so it's okay to yield inside it. - A watcher callback never runs in parallel with itself: if the key to which a watcher is subscribed - is updated while the watcher callback is running, the callback will be invoked again with the new value as - soon as it returns. + A watcher callback is always invoked in a new fiber so it is okay to yield in it. + A watcher callback is never executed in parallel with itself. + If the key is updated while the watcher callback is running, the callback will be invoked again with the new + value as soon as it returns. - Watchers survive reconnect (see ``reconnect_after`` connection option): - all registered watchers are automatically resubscribed as soon as the + Watchers survive reconnection (see ``reconnect_after`` connection option). + All registered watchers are automatically resubscribed when the connection is reestablished. If a remote host supports watchers, the 'watchers' key will be set in connection's ``peer_protocol_features``. + Keep in mind that garbage collection of a watcher handle doesn't result in unregistering the watcher. + It is okay to discard the result of ``box.watch`` if the watcher is never going to be unregistered. + **Example:** Server: @@ -583,7 +582,7 @@ Below is a list of all ``net.box`` functions. .. code-block:: lua conn = net.box.connect(URI) - -- Subscribe to updates of key 'foo'. + -- Subscribe to updates of the 'foo' key. w = conn:watch('foo', function(key, value) assert(key == 'foo') -- do something with value diff --git a/doc/reference/reference_rock/vshard/vshard_pub-sub.rst b/doc/reference/reference_rock/vshard/vshard_pub-sub.rst index d07cb9428..8454f46cb 100644 --- a/doc/reference/reference_rock/vshard/vshard_pub-sub.rst +++ b/doc/reference/reference_rock/vshard/vshard_pub-sub.rst @@ -30,7 +30,7 @@ This prefix is reserved for the built-in events. It means that you cannot create Note that built-in events can't be overridden. It means that the user cannot call -:doc:`box.broadcast('box.id', any_data) `. +:doc:`box.broadcast('box.id', any_data) `. Below is a table of all events that includes the name of the event and its description. @@ -45,7 +45,7 @@ Below is a table of all events that includes the name of the event and its descr * - box.id - Identification of the instance. Changes are particelarly rare. Some values never change or change only once. For example, the UUID of the instance never - changes after the first :doc:`box.cfg `. + changes after the first :doc:`box.cfg `. However, it is not known before ``box.cfg`` is called. The replicaset UUID is unknown until the instance joins a replicaset or boots a new one, but events are supposed to start working before then -- @@ -54,11 +54,11 @@ Below is a table of all events that includes the name of the event and its descr * - box.status - Generic blob about the instance status. There are the most frequently used - and not frequently changed config options and :doc:`box.info ` + and not frequently changed config options and :doc:`box.info ` fields. * - box.election - - All the required parts of :doc:`box.info.election ` + - All the required parts of :doc:`box.info.election ` needed to find out who is the most recent writable leader. * - box.schema @@ -98,7 +98,7 @@ The value for each of the built-in events is written in the following code-block The events are available from the beginning as non-``MP_NIL``. It is necessary for supported local subscriptions. Otherwise, there is no way to detect whether an event is supported at all by this Tarantool version. -If the events are broadcast before :doc:`box.cfg{} `, +If the events are broadcast before :doc:`box.cfg{} `, then the following values are available: .. code-block:: lua From 6b07a34d7d688facbe711b68b9ac47b8666bbd97 Mon Sep 17 00:00:00 2001 From: Kseniia Antonova Date: Mon, 27 Jun 2022 17:38:00 +0300 Subject: [PATCH 06/13] Improve text * check file path * update text * add glossary * fix links --- doc/reference/reference_lua/box_watchers.rst | 7 ++- .../reference_lua/box_watchers/watch.rst | 14 ++++++ doc/reference/reference_lua/index.rst | 1 + doc/reference/reference_lua/net_box.rst | 8 ++-- .../pub-sub.rst} | 48 +++++++++++++++---- doc/reference/reference_rock/vshard/index.rst | 1 - 6 files changed, 60 insertions(+), 19 deletions(-) rename doc/reference/{reference_rock/vshard/vshard_pub-sub.rst => reference_lua/pub-sub.rst} (74%) diff --git a/doc/reference/reference_lua/box_watchers.rst b/doc/reference/reference_lua/box_watchers.rst index 5c8f3cf62..36980d3a7 100644 --- a/doc/reference/reference_lua/box_watchers.rst +++ b/doc/reference/reference_lua/box_watchers.rst @@ -7,7 +7,7 @@ Functions for watchers The ``box`` module contains some functions related to event subscriptions, also known as watchers. The subscriptions are used to inform a client about server-side events. To see the list of the built-in events in Tarantool, -check the :doc:`pub/sub ` section. +check the :doc:`pub/sub ` section. .. glossary:: @@ -21,14 +21,13 @@ check the :doc:`pub/sub ` secti The value is an arbitrary type that can be encoded as MsgPack. To update a state, use the ``box.broadcast()`` function. -A watcher callback is passed the key for which it was registered and the current key data. +A watcher callback is passed the name of the key for which it was registered and the current key data. A watcher callback is always called in a new fiber, so it is okay to yield in it. A watcher callback is never executed in parallel with itself. If the key is updated while the callback is running, the callback will be invoked with the new value as soon as it returns. -The ``watch()`` function returns a watcher handle that can be used to unregister the watcher -when it is no longer needed. +``box.watch`` and ``box.broadcast`` functions can be used before :doc:`box.cfg `. Below is a list of all functions related to watchers. diff --git a/doc/reference/reference_lua/box_watchers/watch.rst b/doc/reference/reference_lua/box_watchers/watch.rst index ff015b9d5..8a50e6b2c 100644 --- a/doc/reference/reference_lua/box_watchers/watch.rst +++ b/doc/reference/reference_lua/box_watchers/watch.rst @@ -7,12 +7,26 @@ box.watch() .. function:: box.watch(key, func) Subscribe to events broadcast by a local host. + It is possible to register more than one watcher for the same key. :param string key: a key name of an event to subscribe to :param function func: a callback to invoke when the key value is updated :return: a watcher handle that can be used to unregister the watcher + A watcher callback is first invoked unconditionally after the watcher registration. + Subsequent invocations are triggered by :doc:`box.broadcast() ` + called on the remote host. + A watcher callback is passed the key for which it was registered and the current key value. + A watcher callback is always invoked in a new fiber so it is okay to yield in it. + A watcher callback is never executed in parallel with itself. + If the key is updated while the watcher callback is running, the callback will be invoked again with the new + value as soon as it returns. + + Note that garbage collection of a watcher handle doesn't result in unregistering the watcher. + It is okay to discard the result of ``box.watch`` if the watcher will never be unregistered. + + **Example:** .. code-block:: lua diff --git a/doc/reference/reference_lua/index.rst b/doc/reference/reference_lua/index.rst index c61c6c37d..720e84bca 100644 --- a/doc/reference/reference_lua/index.rst +++ b/doc/reference/reference_lua/index.rst @@ -56,6 +56,7 @@ This reference covers Tarantool's built-in Lua modules. uri xlog yaml + pub-sub other errcodes debug_facilities diff --git a/doc/reference/reference_lua/net_box.rst b/doc/reference/reference_lua/net_box.rst index 2840eb3ef..fed9327c2 100644 --- a/doc/reference/reference_lua/net_box.rst +++ b/doc/reference/reference_lua/net_box.rst @@ -550,9 +550,9 @@ Below is a list of all ``net.box`` functions. function, which is used for subscribing to events locally. A watcher callback is first invoked unconditionally after the watcher registration. - Subsequent invocations are triggered by :doc:`box.broadcast() ` + Subsequent invocations are triggered by :doc:`box.broadcast() ` called on the remote host. - A watcher callback is passed the name of the key the watcher was subscribed to and the current key value. + A watcher callback is passed the name of the key for which it was registered and the current key data. A watcher callback is always invoked in a new fiber so it is okay to yield in it. A watcher callback is never executed in parallel with itself. If the key is updated while the watcher callback is running, the callback will be invoked again with the new @@ -562,11 +562,11 @@ Below is a list of all ``net.box`` functions. All registered watchers are automatically resubscribed when the connection is reestablished. - If a remote host supports watchers, the 'watchers' key will be set in + If a remote host supports watchers, the ``watchers`` key will be set in the connection's ``peer_protocol_features``. Keep in mind that garbage collection of a watcher handle doesn't result in unregistering the watcher. - It is okay to discard the result of ``box.watch`` if the watcher is never going to be unregistered. + It is okay to discard the result of ``box.watch`` if the watcher will never be unregistered. **Example:** diff --git a/doc/reference/reference_rock/vshard/vshard_pub-sub.rst b/doc/reference/reference_lua/pub-sub.rst similarity index 74% rename from doc/reference/reference_rock/vshard/vshard_pub-sub.rst rename to doc/reference/reference_lua/pub-sub.rst index 8454f46cb..58bf4fb4d 100644 --- a/doc/reference/reference_rock/vshard/vshard_pub-sub.rst +++ b/doc/reference/reference_lua/pub-sub.rst @@ -1,10 +1,38 @@ -.. _vshard-pubsub: +.. _pubsub: Pub/sub system and its events ============================= -Pub/sub is a notification system for Tarantool events. -It is associated with one-time subscriptions. +Overview +-------- + +In Tarantool, :term:`pub/sub ` is a :term:`notification` system for server-side +:term:`events `. + +.. glossary:: + + event + + An event is a state change or a system update update that triggers the action of other systems. + + publisher/subscriber + + A publish/subscribe pattern, also shortened as pub/sub, is a design pattern that consists of + event producers (publishers), event consumers (subscribers or watchers), and event channel. + Publishers communicate with watchers asynchronously by broadcasting events. + They sent events to the pub/sub system. + Then the system delivers received events to all the watchers that react to them. + + notification + + A notification is a message created by one side to inform about the occurence of the event and describe + it to other components. + +The pub/sub pattern is associated with subscriptions. +Each subscription is defined by the certain key. +The main feature of the subscriptions is a one-time action. +It means that if the server generates too many events and a client is too slow, +the server will not allocate additional memory. The system processes the following events: @@ -42,10 +70,10 @@ Below is a table of all events that includes the name of the event and its descr * - Built-in event - Description - * - box.id - - Identification of the instance. Changes are particelarly rare. Some + * - :doc:`box.id ` + - Identification of the instance. Changes are particularly rare. Some values never change or change only once. For example, the UUID of the instance never - changes after the first :doc:`box.cfg `. + changes after the first :doc:`box.cfg `. However, it is not known before ``box.cfg`` is called. The replicaset UUID is unknown until the instance joins a replicaset or boots a new one, but events are supposed to start working before then -- @@ -54,8 +82,8 @@ Below is a table of all events that includes the name of the event and its descr * - box.status - Generic blob about the instance status. There are the most frequently used - and not frequently changed config options and :doc:`box.info ` - fields. + and not frequently changed :doc:`config options ` and + :doc:`box.info ` fields. * - box.election - All the required parts of :doc:`box.info.election ` @@ -95,7 +123,7 @@ The value for each of the built-in events is written in the following code-block MP_STR “version”: MP_UINT schema_version, } -The events are available from the beginning as non-``MP_NIL``. +The events are available from the beginning as non-:ref:`MP_NIL `. It is necessary for supported local subscriptions. Otherwise, there is no way to detect whether an event is supported at all by this Tarantool version. If the events are broadcast before :doc:`box.cfg{} `, @@ -109,7 +137,7 @@ then the following values are available: box.election = {} This way, users can distinguish if an event being not supported -at all or if ``box.cfg{}`` has not beeen called yet. +at all or if ``box.cfg{}`` has not been called yet. Otherwise, they would need to parse the ``_TARANTOOL`` version string locally and the ``peer_version`` in ``net.box``. Usage example diff --git a/doc/reference/reference_rock/vshard/index.rst b/doc/reference/reference_rock/vshard/index.rst index c86b16216..f63cc3803 100644 --- a/doc/reference/reference_rock/vshard/index.rst +++ b/doc/reference/reference_rock/vshard/index.rst @@ -21,4 +21,3 @@ complete ``vshard`` documentation: vshard_quick vshard_ref vshard_api - vshard_pub-sub From abc4741ef819438ef20e5a7fdf33215af83038bf Mon Sep 17 00:00:00 2001 From: Kseniia Antonova Date: Thu, 30 Jun 2022 18:39:19 +0300 Subject: [PATCH 07/13] Update after the review --- doc/reference/reference_lua/box_watchers.rst | 25 ++++++++++++----- .../reference_lua/box_watchers/broadcast.rst | 5 ++++ .../reference_lua/box_watchers/watch.rst | 16 ++--------- doc/reference/reference_lua/net_box.rst | 26 ++++++------------ doc/reference/reference_lua/pub-sub.rst | 27 +++---------------- 5 files changed, 38 insertions(+), 61 deletions(-) diff --git a/doc/reference/reference_lua/box_watchers.rst b/doc/reference/reference_lua/box_watchers.rst index 36980d3a7..3729bfbb5 100644 --- a/doc/reference/reference_lua/box_watchers.rst +++ b/doc/reference/reference_lua/box_watchers.rst @@ -13,7 +13,8 @@ check the :doc:`pub/sub ` section. Watcher A watcher is a :doc:`callback ` that is invoked when a state change occurs. - To create a local watcher, use the ``box.watch()`` function. + To register a local watcher, use the ``box.watch()`` function. + To create a remote watcher, user the ``conn:watch()`` function. State A state is an internally stored key-value pair. @@ -21,14 +22,25 @@ check the :doc:`pub/sub ` section. The value is an arbitrary type that can be encoded as MsgPack. To update a state, use the ``box.broadcast()`` function. +First, a watcher callback is invoked unconditionally after the watcher registration. +Note that it is possible to register more than one watcher for the same key. +After that, :doc:`box.broadcast() ` called on the remote host +triggers all subsequent invocations. + A watcher callback is passed the name of the key for which it was registered and the current key data. -A watcher callback is always called in a new fiber, so it is okay to yield in it. +A watcher callback is always invoked in a new fiber. It means that is is okay to yield in it. A watcher callback is never executed in parallel with itself. -If the key is updated while the callback is running, -the callback will be invoked with the new value as soon as it returns. +If the key is updated while the watcher callback is running, the callback will be invoked again with the new +value as soon as it returns. ``box.watch`` and ``box.broadcast`` functions can be used before :doc:`box.cfg `. +.. note:: + + Keep in mind that garbage collection of a watcher handle doesn't lead to the watcher's destruction. + In this case, the watcher remains registered. + It is okay to discard the result of ``watch`` function if the watcher will never be unregistered. + Below is a list of all functions related to watchers. .. container:: table @@ -45,8 +57,9 @@ Below is a list of all functions related to watchers. * - :doc:`./box_watchers/watch` - Create a local watcher. - To create a remote watcher, see the :ref:`conn:watch() ` function - in the ``net.box`` module. + + * - :ref:`conn:watch() ` + - Create a watcher for the remote host. * - :doc:`./box_watchers/broadcast` - Update a state. diff --git a/doc/reference/reference_lua/box_watchers/broadcast.rst b/doc/reference/reference_lua/box_watchers/broadcast.rst index ac2dfa1ef..b288cf472 100644 --- a/doc/reference/reference_lua/box_watchers/broadcast.rst +++ b/doc/reference/reference_lua/box_watchers/broadcast.rst @@ -13,6 +13,11 @@ box.broadcast() :return: none + **Possible errors:** + + * The value can't be encoded as MSgPack. + * The key refers to ``box.`` system event + **Example:** .. code-block:: lua diff --git a/doc/reference/reference_lua/box_watchers/watch.rst b/doc/reference/reference_lua/box_watchers/watch.rst index 8a50e6b2c..b64c124a0 100644 --- a/doc/reference/reference_lua/box_watchers/watch.rst +++ b/doc/reference/reference_lua/box_watchers/watch.rst @@ -7,25 +7,13 @@ box.watch() .. function:: box.watch(key, func) Subscribe to events broadcast by a local host. - It is possible to register more than one watcher for the same key. :param string key: a key name of an event to subscribe to :param function func: a callback to invoke when the key value is updated - :return: a watcher handle that can be used to unregister the watcher - - A watcher callback is first invoked unconditionally after the watcher registration. - Subsequent invocations are triggered by :doc:`box.broadcast() ` - called on the remote host. - A watcher callback is passed the key for which it was registered and the current key value. - A watcher callback is always invoked in a new fiber so it is okay to yield in it. - A watcher callback is never executed in parallel with itself. - If the key is updated while the watcher callback is running, the callback will be invoked again with the new - value as soon as it returns. - - Note that garbage collection of a watcher handle doesn't result in unregistering the watcher. - It is okay to discard the result of ``box.watch`` if the watcher will never be unregistered. + :return: a watcher handle. The handle consists of one method -- ``unregister()``, which unregisters the watcher. + To read more about watchers, see the `Functions for watchers ` section. **Example:** diff --git a/doc/reference/reference_lua/net_box.rst b/doc/reference/reference_lua/net_box.rst index fed9327c2..0597a12a6 100644 --- a/doc/reference/reference_lua/net_box.rst +++ b/doc/reference/reference_lua/net_box.rst @@ -218,7 +218,7 @@ Below is a list of all ``net.box`` functions. support the specified features, the connection will fail with an error message. With ``required_protocol_features = {'transactions'}``, all connections fail where the server has ``transactions: false``. - + .. container:: table .. list-table:: @@ -242,7 +242,7 @@ Below is a list of all ``net.box`` functions. - IPROTO_FEATURE_ERROR_EXTENSION - 2 and newer * - ``watchers`` - - Requires remote watchers support on the server + - Requires remote :ref:`watchers ` support on the server - IPROTO_FEATURE_WATCHERS - 3 and newer @@ -543,30 +543,20 @@ Below is a list of all ``net.box`` functions. :param string key: a key name of an event to subscribe to :param function func: a callback to invoke when the key value is updated - :return: a watcher handle that can be used to unregister the watcher + :return: a watcher handle. The handle consists of one method -- ``unregister()``, which unregisters the watcher. + + To read more about watchers, see the `Functions for watchers ` section. - Using the ``watch`` method of a net.box connection, you can subscribe to events broadcast by a remote host. The method has the same syntax as the :doc:`box.watch() ` function, which is used for subscribing to events locally. - A watcher callback is first invoked unconditionally after the watcher registration. - Subsequent invocations are triggered by :doc:`box.broadcast() ` - called on the remote host. - A watcher callback is passed the name of the key for which it was registered and the current key data. - A watcher callback is always invoked in a new fiber so it is okay to yield in it. - A watcher callback is never executed in parallel with itself. - If the key is updated while the watcher callback is running, the callback will be invoked again with the new - value as soon as it returns. - - Watchers survive reconnection (see ``reconnect_after`` connection option). + Watchers survive reconnection (see the ``reconnect_after`` connection :ref:`option `). All registered watchers are automatically resubscribed when the connection is reestablished. If a remote host supports watchers, the ``watchers`` key will be set in the - connection's ``peer_protocol_features``. - - Keep in mind that garbage collection of a watcher handle doesn't result in unregistering the watcher. - It is okay to discard the result of ``box.watch`` if the watcher will never be unregistered. + connection ``peer_protocol_features``. + For details, check the :ref:`net.box features table `. **Example:** diff --git a/doc/reference/reference_lua/pub-sub.rst b/doc/reference/reference_lua/pub-sub.rst index 58bf4fb4d..61b1d2284 100644 --- a/doc/reference/reference_lua/pub-sub.rst +++ b/doc/reference/reference_lua/pub-sub.rst @@ -25,7 +25,7 @@ In Tarantool, :term:`pub/sub ` is a :term:`notification` s notification - A notification is a message created by one side to inform about the occurence of the event and describe + A notification is a message created by one side to inform about the occurrence of the event and describe it to other components. The pub/sub pattern is associated with subscriptions. @@ -33,34 +33,15 @@ Each subscription is defined by the certain key. The main feature of the subscriptions is a one-time action. It means that if the server generates too many events and a client is too slow, the server will not allocate additional memory. - -The system processes the following events: - -* ``box.id`` -* ``box.status`` -* ``box.election`` -* ``box.schema`` - In response to each event, the server sends back certain IPROTO fields. Built-in events for pub/sub --------------------------- -One of the most important purposes of the built-in events is :ref:`master discovery `. -It is necessary to know the master node in order to send changes to a correct instance -or to read the most recent data. -In addition, there are more built-in events for other mutable properties such as the change of the leader's -state, its election role and term, the change of the schema version, -and the state of the instance. - -Built-in events have a special naming schema -- theirs names always start with the ``box`` prefix. +Built-in events have a special naming schema -- theirs names always start with the ``box.`` prefix. This prefix is reserved for the built-in events. It means that you cannot create new events with it. -Note that built-in events can't be overridden. -It means that the user cannot call -:doc:`box.broadcast('box.id', any_data) `. - -Below is a table of all events that includes the name of the event and its description. +Below is a table of all built-in events that includes the name of the event and its description. .. container:: table @@ -87,7 +68,7 @@ Below is a table of all events that includes the name of the event and its descr * - box.election - All the required parts of :doc:`box.info.election ` - needed to find out who is the most recent writable leader. + that are necessary to find out who is the most recent writable leader. * - box.schema - Schema-related data. Currently, it contains only the version. From 0d0007050b1bd755f0b3285a3dd83b622230081a Mon Sep 17 00:00:00 2001 From: Kseniia Antonova Date: Thu, 30 Jun 2022 20:31:21 +0300 Subject: [PATCH 08/13] Update after the review --- doc/reference/reference_lua/pub-sub.rst | 72 ++++++++++++++----------- 1 file changed, 42 insertions(+), 30 deletions(-) diff --git a/doc/reference/reference_lua/pub-sub.rst b/doc/reference/reference_lua/pub-sub.rst index 61b1d2284..c6a4d7a26 100644 --- a/doc/reference/reference_lua/pub-sub.rst +++ b/doc/reference/reference_lua/pub-sub.rst @@ -33,7 +33,6 @@ Each subscription is defined by the certain key. The main feature of the subscriptions is a one-time action. It means that if the server generates too many events and a client is too slow, the server will not allocate additional memory. -In response to each event, the server sends back certain IPROTO fields. Built-in events for pub/sub --------------------------- @@ -41,39 +40,30 @@ Built-in events for pub/sub Built-in events have a special naming schema -- theirs names always start with the ``box.`` prefix. This prefix is reserved for the built-in events. It means that you cannot create new events with it. -Below is a table of all built-in events that includes the name of the event and its description. - -.. container:: table +The system processes the following events: - .. list-table:: - :widths: 50 50 +* ``box.id`` +* ``box.status`` +* ``box.election`` +* ``box.schema`` - * - Built-in event - - Description +In response to each event, the server sends back certain IPROTO fields. - * - :doc:`box.id ` - - Identification of the instance. Changes are particularly rare. Some - values never change or change only once. For example, the UUID of the instance never - changes after the first :doc:`box.cfg `. - However, it is not known before ``box.cfg`` is called. - The replicaset UUID is unknown until the instance joins a replicaset or - boots a new one, but events are supposed to start working before then -- - right when the listen launches. The numeric instance ID is known only after - registration. For anonymous replicas, the value is ``0`` until they are officially registered. +box.id +~~~~~~ - * - box.status - - Generic blob about the instance status. There are the most frequently used - and not frequently changed :doc:`config options ` and - :doc:`box.info ` fields. +Contains identification of the instance. +Changes are particularly rare. +Some values never change or change only once. - * - box.election - - All the required parts of :doc:`box.info.election ` - that are necessary to find out who is the most recent writable leader. +The numeric instance ID is known only after registration. +For anonymous replicas, the value is ``0`` until they are officially registered. - * - box.schema - - Schema-related data. Currently, it contains only the version. +The UUID of the instance never changes after the first :doc:`box.cfg `. +The value is unknown before the ``box.cfg`` call. -The value for each of the built-in events is written in the following code-block:. +The replicaset UUID is unknown until the instance joins a replicaset or boots a new one. +The events are supposed to start working before that -- right with the start of the listen. .. code-block:: lua @@ -84,14 +74,29 @@ The value for each of the built-in events is written in the following code-block MP_STR “replicaset_uuid”: MP_UUID box.info.cluster.uuid, } - -- box.status value +box.status +~~~~~~~~~~ + +Contains generic blob about the instance status. +There are the most frequently used and not frequently changed +:doc:`config options ` and :doc:`box.info ` fields. + +.. code-block:: lua + { MP_STR “is_ro”: MP_BOOL box.info.ro, MP_STR “is_ro_cfg”: MP_BOOL box.cfg.read_only, MP_STR “status”: MP_STR box.info.status, } - -- box.election value +box.election +~~~~~~~~~~~~ + +Contains all the required parts of :doc:`box.info.election ` +that are necessary to find out who is the most recent writable leader. + +.. code-block:: lua + { MP_STR “term”: MP_UINT box.info.election.term, MP_STR “role”: MP_STR box.info.election.state, @@ -99,7 +104,14 @@ The value for each of the built-in events is written in the following code-block MP_STR “leader”: MP_UINT box.info.election.leader, } - -- box.schema value +box.schema +~~~~~~~~~~ + +Contains schema-related data. +Currently, it contains only the version. + +.. code-block:: lua + { MP_STR “version”: MP_UINT schema_version, } From a3a22cc3d0141f8ab4d4e4444f548c8395f3b306 Mon Sep 17 00:00:00 2001 From: Kseniia Antonova Date: Mon, 4 Jul 2022 12:59:28 +0300 Subject: [PATCH 09/13] Resolved some comments * Renamed pub/sub section to System events, removed paragraph about pub/sub, moved the event definition to Event watchers section * Removed code-block with empty table for the event values * Fixed some refs and links --- doc/reference/reference_lua/box.rst | 2 +- doc/reference/reference_lua/box_events.rst | 82 +++++++++ .../broadcast.rst | 0 .../box_events/system_events.rst | 126 +++++++++++++ .../{box_watchers => box_events}/watch.rst | 6 + doc/reference/reference_lua/box_watchers.rst | 71 -------- doc/reference/reference_lua/index.rst | 1 - doc/reference/reference_lua/net_box.rst | 10 +- doc/reference/reference_lua/pub-sub.rst | 165 ------------------ 9 files changed, 223 insertions(+), 240 deletions(-) create mode 100644 doc/reference/reference_lua/box_events.rst rename doc/reference/reference_lua/{box_watchers => box_events}/broadcast.rst (100%) create mode 100644 doc/reference/reference_lua/box_events/system_events.rst rename doc/reference/reference_lua/{box_watchers => box_events}/watch.rst (78%) delete mode 100644 doc/reference/reference_lua/box_watchers.rst delete mode 100644 doc/reference/reference_lua/pub-sub.rst diff --git a/doc/reference/reference_lua/box.rst b/doc/reference/reference_lua/box.rst index 218d709ed..efa4cd39e 100644 --- a/doc/reference/reference_lua/box.rst +++ b/doc/reference/reference_lua/box.rst @@ -34,10 +34,10 @@ with ``box``, with no arguments. The ``box`` module contains: box_space box_stat box_tuple - box_watchers box_txn_management box_sql + box_events box_once box_snapshot diff --git a/doc/reference/reference_lua/box_events.rst b/doc/reference/reference_lua/box_events.rst new file mode 100644 index 000000000..624d89244 --- /dev/null +++ b/doc/reference/reference_lua/box_events.rst @@ -0,0 +1,82 @@ +.. _box-watchers: + +Event watchers +============== + +The ``box`` module contains some features related to event subscriptions, also known as :term:`watchers `. +The subscriptions are used to inform a client about server-side :term:`events `. +Each event subscription is defined by a certain key. + +.. glossary:: + + Event + + An event is a state change or a system update that triggers the action of other systems. + To read more about the built-in events in Tarantool, + check the :doc:`system events ` section. + + State + A state is an internally stored key-value pair. + The key is a string. + The value is an arbitrary type that can be encoded as MsgPack. + To update a state, use the ``box.broadcast()`` function. + + Watcher + A watcher is a :doc:`callback ` that is invoked when a state change occurs. + To register a local watcher, use the ``box.watch()`` function. + To create a remote watcher, use the ``watch()`` function from the ``net.box`` module. + Note that it is possible to register more than one watcher for the same key. + +How the watcher works +--------------------- + +First, you register a watcher. +After that, the watcher callback is invoked for the first time. +In this case, the callback is triggered whether or not the key has already been broadcast. +All subsequent invocations are triggered with the :doc:`box.broadcast() ` +called on the remote host. +If a watcher is subscribed for a key that has not been broadcast yet, the callback is triggered only once, +after the registration of the watcher. + +The watcher callback takes two arguments. +The first argument is a name of the key for which it was registered. +The second one contains current key data. +The callback is always invoked in a new fiber. It means that is is okay to yield in it. +A watcher callback is never executed in parallel with itself. +If the key is updated while the watcher callback is running, the callback will be invoked again with the new +value as soon as it returns. + +``box.watch`` and ``box.broadcast`` functions can be used before :doc:`box.cfg `. + +Below is a list of all functions and members related to watchers or events. + +.. container:: table + + .. rst-class:: left-align-column-1 + .. rst-class:: left-align-column-2 + + .. list-table:: + :widths: 25 75 + :header-rows: 1 + + * - Name + - Use + + * - :doc:`./box_events/watch` + - Create a local watcher. + + * - :ref:`conn:watch() ` + - Create a watcher for the remote host. + + * - :doc:`./box_events/broadcast` + - Update a state. + + * - :ref:`˜Built-in events ` + - Predefined events in Tarantool + +.. toctree:: + :hidden: + + box_events/watch + box_events/broadcast + box_events/system_events \ No newline at end of file diff --git a/doc/reference/reference_lua/box_watchers/broadcast.rst b/doc/reference/reference_lua/box_events/broadcast.rst similarity index 100% rename from doc/reference/reference_lua/box_watchers/broadcast.rst rename to doc/reference/reference_lua/box_events/broadcast.rst diff --git a/doc/reference/reference_lua/box_events/system_events.rst b/doc/reference/reference_lua/box_events/system_events.rst new file mode 100644 index 000000000..93f5f98d5 --- /dev/null +++ b/doc/reference/reference_lua/box_events/system_events.rst @@ -0,0 +1,126 @@ +.. _system-events: + +System events +============= + +Predefined events have a special naming schema -- theirs names always start with the reserved ``box.`` prefix. +It means that you cannot create new events with it. + +The system processes the following events: + +* ``box.id`` +* ``box.status`` +* ``box.election`` +* ``box.schema`` + +In response to each event, the server sends back certain ``IPROTO`` fields. + +The events are available from the beginning as non-:ref:`MP_NIL `. +If a watcher subscribes to a system event before it has been broadcast, +it receives an empty table for the event value. + +box.id +~~~~~~ + +Contains :ref:`identification ` of the instance. +Value changes are rare. + +* ``id``: the numeric instance ID is unknown before the registration. + For anonymous replicas, the value is ``0`` until they are officially registered. + +* ``instance_uuid``: the UUID of the instance never changes after the first + :doc:`box.cfg `. + The value is unknown before the ``box.cfg`` call. + +* ``replicaset_uuid``: the value is unknown until the instance joins a replicaset or boots a new one. + The events start working before that -- right with the start of the listen. + +.. code-block:: lua + + -- box.id value + { + MP_STR “id”: MP_UINT; box.info.id, + MP_STR “instance_uuid”: MP_UUID; box.info.uuid, + MP_STR “replicaset_uuid”: MP_UUID box.info.cluster.uuid, + } + +box.status +~~~~~~~~~~ + +Contains generic blob about the instance status. + +* ``is_ro``: :ref:`indicates the read-only mode ` or the ``orphan`` status. +* ``is_ro_cfg``: indicates the :ref:`read_only ` mode for the instance. +* ``status``: shows the status of an instance. + +.. code-block:: lua + + { + MP_STR “is_ro”: MP_BOOL box.info.ro, + MP_STR “is_ro_cfg”: MP_BOOL box.cfg.read_only, + MP_STR “status”: MP_STR box.info.status, + } + +box.election +~~~~~~~~~~~~ + +Contains fields of the :doc:`box.info.election ` +that are necessary to find out the most recent writable leader. + +* ``term``: shows the current election term. +* ``role``: indicates the election state of the node -- ``leader``, ``follower``, or ``candidate``. +* ``is_ro``: :ref:`indicates the read-only mode ` or the ``orphan`` status. +* ``leader``: shows the leader node ID in the current term. + +.. code-block:: lua + + { + MP_STR “term”: MP_UINT box.info.election.term, + MP_STR “role”: MP_STR box.info.election.state, + MP_STR “is_ro”: MP_BOOL box.info.ro, + MP_STR “leader”: MP_UINT box.info.election.leader, + } + +box.schema +~~~~~~~~~~ + +Contains schema-related data. + +* ``version``: shows the schema version. + +.. code-block:: lua + + { + MP_STR “version”: MP_UINT schema_version, + } + +Usage example +------------- + +.. code-block:: lua + + conn = net.box.connect(URI) + -- Subscribe to updates of key 'box.id' + w = conn:watch('box.id', function(key, value) + assert(key == 'box.id') + -- do something with value + end) + -- or to updates of key 'box.status' + w = conn:watch('box.status', function(key, value) + assert(key == 'box.status') + -- do something with value + end) + -- or to updates of key 'box.election' + w = conn:watch('box.election', function(key, value) + assert(key == 'box.election') + -- do something with value + end) + -- or to updates of key 'box.schema' + w = conn:watch('box.schema', function(key, value) + assert(key == 'box.schema') + -- do something with value + end) + -- Unregister the watcher when it's no longer needed. + w:unregister() + + diff --git a/doc/reference/reference_lua/box_watchers/watch.rst b/doc/reference/reference_lua/box_events/watch.rst similarity index 78% rename from doc/reference/reference_lua/box_watchers/watch.rst rename to doc/reference/reference_lua/box_events/watch.rst index b64c124a0..b0b9677d1 100644 --- a/doc/reference/reference_lua/box_watchers/watch.rst +++ b/doc/reference/reference_lua/box_events/watch.rst @@ -15,6 +15,12 @@ box.watch() To read more about watchers, see the `Functions for watchers ` section. + .. note:: + + Keep in mind that garbage collection of a watcher handle doesn't lead to the watcher's destruction. + In this case, the watcher remains registered. + It is okay to discard the result of ``watch`` function if the watcher will never be unregistered. + **Example:** .. code-block:: lua diff --git a/doc/reference/reference_lua/box_watchers.rst b/doc/reference/reference_lua/box_watchers.rst deleted file mode 100644 index 3729bfbb5..000000000 --- a/doc/reference/reference_lua/box_watchers.rst +++ /dev/null @@ -1,71 +0,0 @@ -.. _box-watchers: - --------------------------------------------------------------------------------- -Functions for watchers --------------------------------------------------------------------------------- - -The ``box`` module contains some functions related to event subscriptions, also known as watchers. -The subscriptions are used to inform a client about server-side events. -To see the list of the built-in events in Tarantool, -check the :doc:`pub/sub ` section. - -.. glossary:: - - Watcher - A watcher is a :doc:`callback ` that is invoked when a state change occurs. - To register a local watcher, use the ``box.watch()`` function. - To create a remote watcher, user the ``conn:watch()`` function. - - State - A state is an internally stored key-value pair. - The key is a string. - The value is an arbitrary type that can be encoded as MsgPack. - To update a state, use the ``box.broadcast()`` function. - -First, a watcher callback is invoked unconditionally after the watcher registration. -Note that it is possible to register more than one watcher for the same key. -After that, :doc:`box.broadcast() ` called on the remote host -triggers all subsequent invocations. - -A watcher callback is passed the name of the key for which it was registered and the current key data. -A watcher callback is always invoked in a new fiber. It means that is is okay to yield in it. -A watcher callback is never executed in parallel with itself. -If the key is updated while the watcher callback is running, the callback will be invoked again with the new -value as soon as it returns. - -``box.watch`` and ``box.broadcast`` functions can be used before :doc:`box.cfg `. - -.. note:: - - Keep in mind that garbage collection of a watcher handle doesn't lead to the watcher's destruction. - In this case, the watcher remains registered. - It is okay to discard the result of ``watch`` function if the watcher will never be unregistered. - -Below is a list of all functions related to watchers. - -.. container:: table - - .. rst-class:: left-align-column-1 - .. rst-class:: left-align-column-2 - - .. list-table:: - :widths: 25 75 - :header-rows: 1 - - * - Name - - Use - - * - :doc:`./box_watchers/watch` - - Create a local watcher. - - * - :ref:`conn:watch() ` - - Create a watcher for the remote host. - - * - :doc:`./box_watchers/broadcast` - - Update a state. - -.. toctree:: - :hidden: - - box_watchers/watch - box_watchers/broadcast \ No newline at end of file diff --git a/doc/reference/reference_lua/index.rst b/doc/reference/reference_lua/index.rst index 720e84bca..c61c6c37d 100644 --- a/doc/reference/reference_lua/index.rst +++ b/doc/reference/reference_lua/index.rst @@ -56,7 +56,6 @@ This reference covers Tarantool's built-in Lua modules. uri xlog yaml - pub-sub other errcodes debug_facilities diff --git a/doc/reference/reference_lua/net_box.rst b/doc/reference/reference_lua/net_box.rst index 0597a12a6..835726524 100644 --- a/doc/reference/reference_lua/net_box.rst +++ b/doc/reference/reference_lua/net_box.rst @@ -547,7 +547,7 @@ Below is a list of all ``net.box`` functions. To read more about watchers, see the `Functions for watchers ` section. - The method has the same syntax as the :doc:`box.watch() ` + The method has the same syntax as the :doc:`box.watch() ` function, which is used for subscribing to events locally. Watchers survive reconnection (see the ``reconnect_after`` connection :ref:`option `). @@ -558,6 +558,12 @@ Below is a list of all ``net.box`` functions. connection ``peer_protocol_features``. For details, check the :ref:`net.box features table `. + .. note:: + + Keep in mind that garbage collection of a watcher handle doesn't lead to the watcher's destruction. + In this case, the watcher remains registered. + It is okay to discard the result of ``watch`` function if the watcher will never be unregistered. + **Example:** Server: @@ -575,7 +581,7 @@ Below is a list of all ``net.box`` functions. -- Subscribe to updates of the 'foo' key. w = conn:watch('foo', function(key, value) assert(key == 'foo') - -- do something with value + -- do something with value end) -- Unregister the watcher if it is no longer needed. w:unregister() diff --git a/doc/reference/reference_lua/pub-sub.rst b/doc/reference/reference_lua/pub-sub.rst deleted file mode 100644 index c6a4d7a26..000000000 --- a/doc/reference/reference_lua/pub-sub.rst +++ /dev/null @@ -1,165 +0,0 @@ -.. _pubsub: - -Pub/sub system and its events -============================= - -Overview --------- - -In Tarantool, :term:`pub/sub ` is a :term:`notification` system for server-side -:term:`events `. - -.. glossary:: - - event - - An event is a state change or a system update update that triggers the action of other systems. - - publisher/subscriber - - A publish/subscribe pattern, also shortened as pub/sub, is a design pattern that consists of - event producers (publishers), event consumers (subscribers or watchers), and event channel. - Publishers communicate with watchers asynchronously by broadcasting events. - They sent events to the pub/sub system. - Then the system delivers received events to all the watchers that react to them. - - notification - - A notification is a message created by one side to inform about the occurrence of the event and describe - it to other components. - -The pub/sub pattern is associated with subscriptions. -Each subscription is defined by the certain key. -The main feature of the subscriptions is a one-time action. -It means that if the server generates too many events and a client is too slow, -the server will not allocate additional memory. - -Built-in events for pub/sub ---------------------------- - -Built-in events have a special naming schema -- theirs names always start with the ``box.`` prefix. -This prefix is reserved for the built-in events. It means that you cannot create new events with it. - -The system processes the following events: - -* ``box.id`` -* ``box.status`` -* ``box.election`` -* ``box.schema`` - -In response to each event, the server sends back certain IPROTO fields. - -box.id -~~~~~~ - -Contains identification of the instance. -Changes are particularly rare. -Some values never change or change only once. - -The numeric instance ID is known only after registration. -For anonymous replicas, the value is ``0`` until they are officially registered. - -The UUID of the instance never changes after the first :doc:`box.cfg `. -The value is unknown before the ``box.cfg`` call. - -The replicaset UUID is unknown until the instance joins a replicaset or boots a new one. -The events are supposed to start working before that -- right with the start of the listen. - -.. code-block:: lua - - -- box.id value - { - MP_STR “id”: MP_UINT; box.info.id, - MP_STR “instance_uuid”: MP_UUID; box.info.uuid, - MP_STR “replicaset_uuid”: MP_UUID box.info.cluster.uuid, - } - -box.status -~~~~~~~~~~ - -Contains generic blob about the instance status. -There are the most frequently used and not frequently changed -:doc:`config options ` and :doc:`box.info ` fields. - -.. code-block:: lua - - { - MP_STR “is_ro”: MP_BOOL box.info.ro, - MP_STR “is_ro_cfg”: MP_BOOL box.cfg.read_only, - MP_STR “status”: MP_STR box.info.status, - } - -box.election -~~~~~~~~~~~~ - -Contains all the required parts of :doc:`box.info.election ` -that are necessary to find out who is the most recent writable leader. - -.. code-block:: lua - - { - MP_STR “term”: MP_UINT box.info.election.term, - MP_STR “role”: MP_STR box.info.election.state, - MP_STR “is_ro”: MP_BOOL box.info.ro, - MP_STR “leader”: MP_UINT box.info.election.leader, - } - -box.schema -~~~~~~~~~~ - -Contains schema-related data. -Currently, it contains only the version. - -.. code-block:: lua - - { - MP_STR “version”: MP_UINT schema_version, - } - -The events are available from the beginning as non-:ref:`MP_NIL `. -It is necessary for supported local subscriptions. -Otherwise, there is no way to detect whether an event is supported at all by this Tarantool version. -If the events are broadcast before :doc:`box.cfg{} `, -then the following values are available: - -.. code-block:: lua - - box.id = {} - box.schema = {} - box.status = {} - box.election = {} - -This way, users can distinguish if an event being not supported -at all or if ``box.cfg{}`` has not been called yet. -Otherwise, they would need to parse the ``_TARANTOOL`` version string locally and the ``peer_version`` in ``net.box``. - -Usage example -------------- - -.. code-block:: lua - - conn = net.box.connect(URI) - -- Subscribe to updates of key 'box.id' - w = conn:watch('box.id', function(key, value) - assert(key == 'box.id') - -- do something with value - end) - -- or to updates of key 'box.status' - w = conn:watch('box.status', function(key, value) - assert(key == 'box.status') - -- do something with value - end) - -- or to updates of key 'box.election' - w = conn:watch('box.election', function(key, value) - assert(key == 'box.election') - -- do something with value - end) - -- or to updates of key 'box.schema' - w = conn:watch('box.schema', function(key, value) - assert(key == 'box.schema') - -- do something with value - end) - -- Unregister the watcher when it's no longer needed. - w:unregister() - - From b35727531b8d91272a9a053bf47a86fd27ea9510 Mon Sep 17 00:00:00 2001 From: Kseniia Antonova Date: Wed, 27 Jul 2022 13:27:05 +0300 Subject: [PATCH 10/13] Update files after the review --- doc/reference/reference_lua/box_events.rst | 6 ++-- .../reference_lua/box_events/broadcast.rst | 4 +-- .../box_events/system_events.rst | 35 ++++++++----------- .../reference_lua/box_events/watch.rst | 18 ++++++---- doc/reference/reference_lua/net_box.rst | 17 +++++---- 5 files changed, 43 insertions(+), 37 deletions(-) diff --git a/doc/reference/reference_lua/box_events.rst b/doc/reference/reference_lua/box_events.rst index 624d89244..40b4b6d5f 100644 --- a/doc/reference/reference_lua/box_events.rst +++ b/doc/reference/reference_lua/box_events.rst @@ -4,7 +4,7 @@ Event watchers ============== The ``box`` module contains some features related to event subscriptions, also known as :term:`watchers `. -The subscriptions are used to inform a client about server-side :term:`events `. +The subscriptions are used to inform a client about the server-side :term:`events `. Each event subscription is defined by a certain key. .. glossary:: @@ -41,7 +41,7 @@ after the registration of the watcher. The watcher callback takes two arguments. The first argument is a name of the key for which it was registered. The second one contains current key data. -The callback is always invoked in a new fiber. It means that is is okay to yield in it. +The callback is always invoked in a new fiber. It means that is allowed to yield in it. A watcher callback is never executed in parallel with itself. If the key is updated while the watcher callback is running, the callback will be invoked again with the new value as soon as it returns. @@ -71,7 +71,7 @@ Below is a list of all functions and members related to watchers or events. * - :doc:`./box_events/broadcast` - Update a state. - * - :ref:`˜Built-in events ` + * - :ref:`Built-in events ` - Predefined events in Tarantool .. toctree:: diff --git a/doc/reference/reference_lua/box_events/broadcast.rst b/doc/reference/reference_lua/box_events/broadcast.rst index b288cf472..74c94dac0 100644 --- a/doc/reference/reference_lua/box_events/broadcast.rst +++ b/doc/reference/reference_lua/box_events/broadcast.rst @@ -22,7 +22,7 @@ box.broadcast() .. code-block:: lua - -- Broadcast value 123 for the 'foo' key. - box.broadcast('foo', 123) + -- Broadcast value 42 for the 'foo' key. + box.broadcast('foo', 42) diff --git a/doc/reference/reference_lua/box_events/system_events.rst b/doc/reference/reference_lua/box_events/system_events.rst index 93f5f98d5..09701f5f3 100644 --- a/doc/reference/reference_lua/box_events/system_events.rst +++ b/doc/reference/reference_lua/box_events/system_events.rst @@ -19,6 +19,12 @@ The events are available from the beginning as non-:ref:`MP_NIL ` section. + To read more about watchers, see the :ref:`Functions for watchers ` section. .. note:: @@ -25,13 +25,19 @@ box.watch() .. code-block:: lua - -- Broadcast value 123 for the 'foo' key. - box.broadcast('foo', 123) + -- Broadcast value 42 for the 'foo' key. + box.broadcast('foo', 42) + + local log = require('log') -- Subscribe to updates of the 'foo' key. - w = box.watch('foo', function(key, value) + local w = box.watch('foo', function(key, value) assert(key == 'foo') - -- do something with value + log.info("The box.id value is '%d'", value) end) - -- Unregister the watcher when it is no longer needed. + + If you don't need the watcher anymore, you can unregister it using the command below: + + .. code-block:: lua + w:unregister() diff --git a/doc/reference/reference_lua/net_box.rst b/doc/reference/reference_lua/net_box.rst index 835726524..ec1909c07 100644 --- a/doc/reference/reference_lua/net_box.rst +++ b/doc/reference/reference_lua/net_box.rst @@ -1,4 +1,4 @@ -.. _net_box-module: +zc.. _net_box-module: -------------------------------------------------------------------------------- Module net.box @@ -545,7 +545,7 @@ Below is a list of all ``net.box`` functions. :param function func: a callback to invoke when the key value is updated :return: a watcher handle. The handle consists of one method -- ``unregister()``, which unregisters the watcher. - To read more about watchers, see the `Functions for watchers ` section. + To read more about watchers, see the :ref:`Functions for watchers ` section. The method has the same syntax as the :doc:`box.watch() ` function, which is used for subscribing to events locally. @@ -570,20 +570,25 @@ Below is a list of all ``net.box`` functions. .. code-block:: lua - -- Broadcast value 123 for the key 'foo'. - box.broadcast('foo', 123) + -- Broadcast value 42 for the 'foo' key. + box.broadcast('foo', 42) Client: .. code-block:: lua conn = net.box.connect(URI) + local log = require('log') -- Subscribe to updates of the 'foo' key. w = conn:watch('foo', function(key, value) assert(key == 'foo') - -- do something with value + log.info("The box.id value is '%d'", value) end) - -- Unregister the watcher if it is no longer needed. + + If you don't need the watcher anymore, you can unregister it using the command below: + + .. code-block:: lua + w:unregister() .. _conn-timeout: From f2b8dbbac4614b2dc7635b138f7c68494355793b Mon Sep 17 00:00:00 2001 From: Patience Daur Date: Fri, 5 Aug 2022 10:36:41 +0300 Subject: [PATCH 11/13] Proofread --- doc/reference/reference_lua/box_events.rst | 12 ++++++------ doc/reference/reference_lua/box_events/broadcast.rst | 6 +++--- .../reference_lua/box_events/system_events.rst | 4 ++-- doc/reference/reference_lua/box_events/watch.rst | 11 +++++++++-- 4 files changed, 20 insertions(+), 13 deletions(-) diff --git a/doc/reference/reference_lua/box_events.rst b/doc/reference/reference_lua/box_events.rst index 40b4b6d5f..4e3dc9cc5 100644 --- a/doc/reference/reference_lua/box_events.rst +++ b/doc/reference/reference_lua/box_events.rst @@ -4,7 +4,7 @@ Event watchers ============== The ``box`` module contains some features related to event subscriptions, also known as :term:`watchers `. -The subscriptions are used to inform a client about the server-side :term:`events `. +The subscriptions are used to inform the client about server-side :term:`events `. Each event subscription is defined by a certain key. .. glossary:: @@ -12,7 +12,7 @@ Each event subscription is defined by a certain key. Event An event is a state change or a system update that triggers the action of other systems. - To read more about the built-in events in Tarantool, + To read more about built-in events in Tarantool, check the :doc:`system events ` section. State @@ -33,22 +33,22 @@ How the watcher works First, you register a watcher. After that, the watcher callback is invoked for the first time. In this case, the callback is triggered whether or not the key has already been broadcast. -All subsequent invocations are triggered with the :doc:`box.broadcast() ` +All subsequent invocations are triggered with :doc:`box.broadcast() ` called on the remote host. If a watcher is subscribed for a key that has not been broadcast yet, the callback is triggered only once, after the registration of the watcher. The watcher callback takes two arguments. -The first argument is a name of the key for which it was registered. +The first argument is the name of the key for which it was registered. The second one contains current key data. -The callback is always invoked in a new fiber. It means that is allowed to yield in it. +The callback is always invoked in a new fiber. It means that it is allowed to yield in it. A watcher callback is never executed in parallel with itself. If the key is updated while the watcher callback is running, the callback will be invoked again with the new value as soon as it returns. ``box.watch`` and ``box.broadcast`` functions can be used before :doc:`box.cfg `. -Below is a list of all functions and members related to watchers or events. +Below is a list of all functions and pages related to watchers or events. .. container:: table diff --git a/doc/reference/reference_lua/box_events/broadcast.rst b/doc/reference/reference_lua/box_events/broadcast.rst index 74c94dac0..45aa91e17 100644 --- a/doc/reference/reference_lua/box_events/broadcast.rst +++ b/doc/reference/reference_lua/box_events/broadcast.rst @@ -8,15 +8,15 @@ box.broadcast() Update the value of a particular key and notify all key watchers of the update. - :param string key: a key name of an event to subscribe to + :param string key: key name of the event to subscribe to :param value: any data that can be encoded in MsgPack :return: none **Possible errors:** - * The value can't be encoded as MSgPack. - * The key refers to ``box.`` system event + * The value can't be encoded as MsgPack. + * The key refers to a ``box.`` system event **Example:** diff --git a/doc/reference/reference_lua/box_events/system_events.rst b/doc/reference/reference_lua/box_events/system_events.rst index 09701f5f3..e36219393 100644 --- a/doc/reference/reference_lua/box_events/system_events.rst +++ b/doc/reference/reference_lua/box_events/system_events.rst @@ -52,7 +52,7 @@ Value changes are rare. box.status ~~~~~~~~~~ -Contains generic blob about the instance status. +Contains generic information about the instance status. * ``is_ro``: :ref:`indicates the read-only mode ` or the ``orphan`` status. * ``is_ro_cfg``: indicates the :ref:`read_only ` mode for the instance. @@ -69,7 +69,7 @@ Contains generic blob about the instance status. box.election ~~~~~~~~~~~~ -Contains fields of the :doc:`box.info.election ` +Contains fields of :doc:`box.info.election ` that are necessary to find out the most recent writable leader. * ``term``: shows the current election term. diff --git a/doc/reference/reference_lua/box_events/watch.rst b/doc/reference/reference_lua/box_events/watch.rst index 9c6641b03..52a9a10e2 100644 --- a/doc/reference/reference_lua/box_events/watch.rst +++ b/doc/reference/reference_lua/box_events/watch.rst @@ -8,8 +8,8 @@ box.watch() Subscribe to events broadcast by a local host. - :param string key: a key name of an event to subscribe to - :param function func: a callback to invoke when the key value is updated + :param string key: key name of the event to subscribe to + :param function func: callback to invoke when the key value is updated :return: a watcher handle. The handle consists of one method -- ``unregister()``, which unregisters the watcher. @@ -23,11 +23,18 @@ box.watch() **Example:** + Server: + .. code-block:: lua -- Broadcast value 42 for the 'foo' key. box.broadcast('foo', 42) + Client: + + .. code-block:: lua + + conn = require('net.box').connect(URI) local log = require('log') -- Subscribe to updates of the 'foo' key. local w = box.watch('foo', function(key, value) From 7d721e30613be2d6201dc27c3804e807776b754e Mon Sep 17 00:00:00 2001 From: Dia Patience Daur Date: Fri, 5 Aug 2022 11:13:48 +0300 Subject: [PATCH 12/13] Correct an article --- doc/reference/reference_lua/box_events.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/reference/reference_lua/box_events.rst b/doc/reference/reference_lua/box_events.rst index 4e3dc9cc5..60a533040 100644 --- a/doc/reference/reference_lua/box_events.rst +++ b/doc/reference/reference_lua/box_events.rst @@ -27,8 +27,8 @@ Each event subscription is defined by a certain key. To create a remote watcher, use the ``watch()`` function from the ``net.box`` module. Note that it is possible to register more than one watcher for the same key. -How the watcher works ---------------------- +How a watcher works +------------------- First, you register a watcher. After that, the watcher callback is invoked for the first time. From 9611be42b96928d55b8512dbb727bae311bdabca Mon Sep 17 00:00:00 2001 From: Dia Patience Daur Date: Fri, 5 Aug 2022 11:34:10 +0300 Subject: [PATCH 13/13] Fix build errors --- doc/reference/reference_lua/box_events/system_events.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/doc/reference/reference_lua/box_events/system_events.rst b/doc/reference/reference_lua/box_events/system_events.rst index e36219393..8dfaf0c7f 100644 --- a/doc/reference/reference_lua/box_events/system_events.rst +++ b/doc/reference/reference_lua/box_events/system_events.rst @@ -40,7 +40,7 @@ Value changes are rare. * ``replicaset_uuid``: the value is unknown until the instance joins a replicaset or boots a new one. -.. code-block:: lua +.. code-block:: none -- box.id value { @@ -58,7 +58,7 @@ Contains generic information about the instance status. * ``is_ro_cfg``: indicates the :ref:`read_only ` mode for the instance. * ``status``: shows the status of an instance. -.. code-block:: lua +.. code-block:: none { MP_STR “is_ro”: MP_BOOL box.info.ro, @@ -77,7 +77,7 @@ that are necessary to find out the most recent writable leader. * ``is_ro``: :ref:`indicates the read-only mode ` or the ``orphan`` status. * ``leader``: shows the leader node ID in the current term. -.. code-block:: lua +.. code-block:: none { MP_STR “term”: MP_UINT box.info.election.term, @@ -93,7 +93,7 @@ Contains schema-related data. * ``version``: shows the schema version. -.. code-block:: lua +.. code-block:: none { MP_STR “version”: MP_UINT schema_version,