API to read event data without making a subscription #6493

Gerold103 · 2021-10-05T21:02:12Z

The ticket #6257 defines subscriptions. The other ticket #6260 defines built-in events available out of the box. One of the features of the subs is that they work without any grants. At least the built-in ones.

It might be useful to have an API to get data by a subscription key without making an actual subscription. It could be used by clients who are not able or don't want to process async events. They might want to poll the data. With the help of such API they wouldn't need to implement stored functions to get stuff like election state, some cfg options, and whatever will be exposed along.

It is possible to emulate it by sending subscribe + unsubscribe requests right after each other, but that means more network data, and more load on the server to create and drop the subscription even though it wasn't really needed.

unera · 2023-02-28T10:45:25Z

We could add a flag to IPROTO_WATCH request.

UNSUBSCRIBE: true

PLACE	TYPE	VALUE
header	IPROTO_REQUEST_TYPE	IPROTO_WATCH
body	IPROTO_EVENT_KEY	MP_STR
body	IPROTO_WATCH_UNSUBSCRIBE	MP_BOOL

If the flag is true, tarantool doesn't add the client to the subscriber's list.

Cases:

if the subscriber's list has already contained the client, it means that a client should stay there.

So algo:

IPROTO_WATCH[unsubscribe = true] -> return state[IPROTO_EVENT_KEY]

locker · 2023-05-12T12:27:40Z

Design document

Straightforward line-by-line rewrite of the local and remote watcher tests from diff to luatest. Needed to add test cases for watch_once. Needed for tarantool#6493 NO_DOC=test NO_CHANGELOG=test

@TarantoolBot

Part of tarantool#6493 @TarantoolBot document Title: Document `box.watch_once()` The function takes a notification key and returns the value currently associated with it. ```yaml tarantool> box.watch_once('foo') --- ... tarantool> box.broadcast('foo', {a = 1, b = 2}) --- ... tarantool> box.watch_once('foo') --- - {'a': 1, 'b': 2} ... ``` The new function can be used instead of `box.watch()` in case the caller only needs to retrieve the current value without subscribing to future changes.

@TarantoolBot

Part of tarantool#6493 @TarantoolBot document Title: Document `IPROTO_WATCH_ONCE` The new request type `IPROTO_WATCH_ONCE` has code 77 and can be used to synchronously fetch the value currently associated with a specified notification key. You can use it instead of `IPROTO_WATCH` if you just need to get the current value without subscribing to future changes. The new request is a standard synchronous request. Like any other synchronous request (for example, `IPROTO_SELECT`), it takes a sync number (`IPROTO_SYNC`) and optionally the desired schema version (`IPROTO_SCHEMA_VERSION`) in the header. The same sync number and the actual schema version are returned in the response header. The request body is supposed to contain a single key `IPROTO_EVENT_KEY` (which is also used by `IPROTO_WATCH`) with a string value that stores the notification key of interest. The actual value is returned in the response body, in the first entry of the `IPROTO_DATA` array. If there's no value associated with a notification key (the key has never been broadcast or was last set to nil), the `IPROTO_DATA` array will be empty. (Note that `IPROTO_DATA` is also used by most other synchronous requests. For example, `IPROTO_SELECT` returns the selected tuple array in it.) For example, suppose a key was broadcast with the following command on the server: ```Lua box.broadcast('foo', {1, 2, 3}) ``` Then `IPROTO_WATCH_ONCE` for `IPROTO_EVENT_KEY` equal to 'foo' will return `IPROTO_DATA' equal to `[[1, 2, 3]]` (an array of one entry containing the current value). If the key didn't exist or was set to nil with ```Lua box.broadcast('foo', nil) ``` then `IPROTO_WATCH_ONCE` would return `IPROTO_DATA` equal to `[]` (an empty array). The request shouldn't normally fail. It may fail only on some sort of system error (out of memory; socket error), on schema version mismatch, or on invalid input. Like `IPROTO_WATCH`, the new request doesn't require authentication. If a server supports the `IPROTO_WATCH_ONCE` request, it'll set the `IPROTO_FEATURE_WATCH_ONCE = 6` bit in the protocol feature mask and report the protocol version >= 6 in response to `IPROTO_ID`.

Duplicating the net.box method constant enum in C and Lua is inconvenient and error-prone. Let's export it to Lua automatically, like we do with IPROTO constants. This will simplify addition of new net.box methods (for example, WATCH_ONCE). While we are at it, - move NETBOX_SELECT_WITH_POS closer to NETBOX_SELECT because they are related so better be defined together; - replace spaces with tabs in the encode/decode method tables for consistency. Needed for tarantool#6493 NO_DOC=refactoring NO_CHANGELOG=refactoring

@TarantoolBot

Closes tarantool#6493 @TarantoolBot document Title: Document `conn:watch_once()` net.box connection method The new method takes a notification key and returns the value currently associated with it. For example, let's assume that a Tarantool server was started with the following script: ```lua box.cfg{listen = 3301} box.broadcast('foo', {1, 2, 3}) ``` Then the `conn:watch_once()` method would yield the following results: ```yaml tarantool> conn = require('net.box').connect(3301) --- ... tarantool> conn:watch_once('foo') --- - [1, 2, 3] ... tarantool> conn:watch_once('bar') --- - null ... ``` The new method can be used instead of `conn:watch()` in case the caller only needs to retrieve the current associated with a notification key value without subscribing to future changes. The method can also take a net.box options table as a second argument. It supports all the standard request options: `is_async`, `return_raw`, `timeout`, and others. They work exactly in the same way as with other net.box method, for example `conn:call`. For example, ```lua local future = conn:watch_once('foo', {is_async = true}) future:wait_result() local obj = conn:watch_once('foo', {return_raw = true}) require('msgpack').is_object(obj) ``` Like `conn:watch()`, the new method doesn't require authentication. Like `conn:watch()`, the new method can be executed in a stream (see `conn:new_stream()`), but it isn't streamlined (i.e. calling it as a stream method has the same effect as calling it as a connection method). The net.box connection will set `conn.peer_protocol_features.watch_once` to true if the remote end supports `conn:watch_once()`. The new method is implemented using the `IPROTO_WATCH_ONCE` request.

@TarantoolBot

Part of tarantool#6493 @TarantoolBot document Title: Document `IPROTO_WATCH_ONCE` The new request type `IPROTO_WATCH_ONCE` has code 77 and can be used to synchronously fetch the value currently associated with a specified notification key. You can use it instead of `IPROTO_WATCH` if you just need to get the current value without subscribing to future changes. The new request is a standard synchronous request. Like any other synchronous request (for example, `IPROTO_SELECT`), it takes a sync number (`IPROTO_SYNC`) and optionally the desired schema version (`IPROTO_SCHEMA_VERSION`) in the header. The same sync number and the actual schema version are returned in the response header. The request body is supposed to contain a single key `IPROTO_EVENT_KEY` (which is also used by `IPROTO_WATCH`) with a string value that stores the notification key of interest. The actual value is returned in the response body, in the first entry of the `IPROTO_DATA` array. If there's no value associated with a notification key (the key has never been broadcast or was last set to nil), the `IPROTO_DATA` array will be empty. (Note that `IPROTO_DATA` is also used by most other synchronous requests. For example, `IPROTO_SELECT` returns the selected tuple array in it.) For example, suppose a key was broadcast with the following command on the server: ```Lua box.broadcast('foo', {1, 2, 3}) ``` Then `IPROTO_WATCH_ONCE` for `IPROTO_EVENT_KEY` equal to 'foo' will return `IPROTO_DATA' equal to `[[1, 2, 3]]` (an array of one entry containing the current value). If the key didn't exist or was set to nil with ```Lua box.broadcast('foo', nil) ``` then `IPROTO_WATCH_ONCE` would return `IPROTO_DATA` equal to `[]` (an empty array). The request shouldn't normally fail. It may fail only on some sort of system error (out of memory; socket error), on schema version mismatch, or on invalid input. Like `IPROTO_WATCH`, the new request doesn't require authentication. Like `IPROTO_WATCH`, the new request can't processed in a stream (`IPROTO_STREAM_ID` must not be set in the request header). If a server supports the `IPROTO_WATCH_ONCE` request, it'll set the `IPROTO_FEATURE_WATCH_ONCE = 6` bit in the protocol feature mask and report the protocol version >= 6 in response to `IPROTO_ID`.

Duplicating the net.box method constant enum in C and Lua is inconvenient and error-prone. Let's export it to Lua automatically, like we do with IPROTO constants. This will simplify addition of new net.box methods (for example, WATCH_ONCE). While we are at it, - move NETBOX_SELECT_WITH_POS closer to NETBOX_SELECT because they are related so better be defined together; - replace spaces with tabs in the encode/decode method tables for consistency. Needed for tarantool#6493 NO_DOC=refactoring NO_CHANGELOG=refactoring

@TarantoolBot

Closes tarantool#6493 @TarantoolBot document Title: Document `conn:watch_once()` net.box connection method The new method takes a notification key and returns the value currently associated with it. For example, let's assume that a Tarantool server was started with the following script: ```lua box.cfg{listen = 3301} box.broadcast('foo', {1, 2, 3}) ``` Then the `conn:watch_once()` method would yield the following results: ```yaml tarantool> conn = require('net.box').connect(3301) --- ... tarantool> conn:watch_once('foo') --- - [1, 2, 3] ... tarantool> conn:watch_once('bar') --- - null ... ``` The new method can be used instead of `conn:watch()` in case the caller only needs to retrieve the current associated with a notification key value without subscribing to future changes. The method can also take a net.box options table as a second argument. It supports all the standard request options: `is_async`, `return_raw`, `timeout`, and others. They work exactly in the same way as with other net.box method, for example `conn:call`. For example, ```lua local future = conn:watch_once('foo', {is_async = true}) future:wait_result() local obj = conn:watch_once('foo', {return_raw = true}) require('msgpack').is_object(obj) ``` Like `conn:watch()`, the new method doesn't require authentication. Like `conn:watch()`, the new method can be executed in a stream (see `conn:new_stream()`), but it isn't streamlined (i.e. calling it as a stream method has the same effect as calling it as a connection method). The net.box connection will set `conn.peer_protocol_features.watch_once` to true if the remote end supports `conn:watch_once()`. The new method is implemented using the `IPROTO_WATCH_ONCE` request.

Straightforward line-by-line rewrite of the local and remote watcher tests from diff to luatest. Needed to add test cases for watch_once. Needed for #6493 NO_DOC=test NO_CHANGELOG=test

@TarantoolBot

Part of #6493 @TarantoolBot document Title: Document `box.watch_once()` The function takes a notification key and returns the value currently associated with it. ```yaml tarantool> box.watch_once('foo') --- ... tarantool> box.broadcast('foo', {a = 1, b = 2}) --- ... tarantool> box.watch_once('foo') --- - {'a': 1, 'b': 2} ... ``` The new function can be used instead of `box.watch()` in case the caller only needs to retrieve the current value without subscribing to future changes.

@TarantoolBot

Part of #6493 @TarantoolBot document Title: Document `IPROTO_WATCH_ONCE` The new request type `IPROTO_WATCH_ONCE` has code 77 and can be used to synchronously fetch the value currently associated with a specified notification key. You can use it instead of `IPROTO_WATCH` if you just need to get the current value without subscribing to future changes. The new request is a standard synchronous request. Like any other synchronous request (for example, `IPROTO_SELECT`), it takes a sync number (`IPROTO_SYNC`) and optionally the desired schema version (`IPROTO_SCHEMA_VERSION`) in the header. The same sync number and the actual schema version are returned in the response header. The request body is supposed to contain a single key `IPROTO_EVENT_KEY` (which is also used by `IPROTO_WATCH`) with a string value that stores the notification key of interest. The actual value is returned in the response body, in the first entry of the `IPROTO_DATA` array. If there's no value associated with a notification key (the key has never been broadcast or was last set to nil), the `IPROTO_DATA` array will be empty. (Note that `IPROTO_DATA` is also used by most other synchronous requests. For example, `IPROTO_SELECT` returns the selected tuple array in it.) For example, suppose a key was broadcast with the following command on the server: ```Lua box.broadcast('foo', {1, 2, 3}) ``` Then `IPROTO_WATCH_ONCE` for `IPROTO_EVENT_KEY` equal to 'foo' will return `IPROTO_DATA' equal to `[[1, 2, 3]]` (an array of one entry containing the current value). If the key didn't exist or was set to nil with ```Lua box.broadcast('foo', nil) ``` then `IPROTO_WATCH_ONCE` would return `IPROTO_DATA` equal to `[]` (an empty array). The request shouldn't normally fail. It may fail only on some sort of system error (out of memory; socket error), on schema version mismatch, or on invalid input. Like `IPROTO_WATCH`, the new request doesn't require authentication. Like `IPROTO_WATCH`, the new request can't processed in a stream (`IPROTO_STREAM_ID` must not be set in the request header). If a server supports the `IPROTO_WATCH_ONCE` request, it'll set the `IPROTO_FEATURE_WATCH_ONCE = 6` bit in the protocol feature mask and report the protocol version >= 6 in response to `IPROTO_ID`.

Duplicating the net.box method constant enum in C and Lua is inconvenient and error-prone. Let's export it to Lua automatically, like we do with IPROTO constants. This will simplify addition of new net.box methods (for example, WATCH_ONCE). While we are at it, - move NETBOX_SELECT_WITH_POS closer to NETBOX_SELECT because they are related so better be defined together; - replace spaces with tabs in the encode/decode method tables for consistency. Needed for #6493 NO_DOC=refactoring NO_CHANGELOG=refactoring

@TarantoolBot

Closes #6493 @TarantoolBot document Title: Document `conn:watch_once()` net.box connection method The new method takes a notification key and returns the value currently associated with it. For example, let's assume that a Tarantool server was started with the following script: ```lua box.cfg{listen = 3301} box.broadcast('foo', {1, 2, 3}) ``` Then the `conn:watch_once()` method would yield the following results: ```yaml tarantool> conn = require('net.box').connect(3301) --- ... tarantool> conn:watch_once('foo') --- - [1, 2, 3] ... tarantool> conn:watch_once('bar') --- - null ... ``` The new method can be used instead of `conn:watch()` in case the caller only needs to retrieve the current associated with a notification key value without subscribing to future changes. The method can also take a net.box options table as a second argument. It supports all the standard request options: `is_async`, `return_raw`, `timeout`, and others. They work exactly in the same way as with other net.box method, for example `conn:call`. For example, ```lua local future = conn:watch_once('foo', {is_async = true}) future:wait_result() local obj = conn:watch_once('foo', {return_raw = true}) require('msgpack').is_object(obj) ``` Like `conn:watch()`, the new method doesn't require authentication. Like `conn:watch()`, the new method can be executed in a stream (see `conn:new_stream()`), but it isn't streamlined (i.e. calling it as a stream method has the same effect as calling it as a connection method). The net.box connection will set `conn.peer_protocol_features.watch_once` to true if the remote end supports `conn:watch_once()`. The new method is implemented using the `IPROTO_WATCH_ONCE` request.

Gerold103 added feature A new functionality app incoming labels Oct 5, 2021

sergos added the teamC label Oct 8, 2021

kyukhin removed the incoming label Dec 9, 2021

Mons removed the app label Feb 21, 2023

Mons assigned unera Feb 21, 2023

unera removed their assignment Feb 28, 2023

locker self-assigned this May 10, 2023

locker added the 3.0 Target is 3.0 and all newer release/master branches label May 12, 2023

kyukhin removed the teamC label May 23, 2023

locker mentioned this issue Jun 6, 2023

Introduce API to read a state value without subscribing (watch once) #8741

Merged

locker closed this as completed in #8741 Jun 13, 2023

locker added a commit that referenced this issue Jun 13, 2023

test: migrate watcher tests to luatest

2e936e4

Straightforward line-by-line rewrite of the local and remote watcher tests from diff to luatest. Needed to add test cases for watch_once. Needed for #6493 NO_DOC=test NO_CHANGELOG=test

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

API to read event data without making a subscription #6493

API to read event data without making a subscription #6493

Gerold103 commented Oct 5, 2021 •

edited by locker

unera commented Feb 28, 2023

locker commented May 12, 2023

API to read event data without making a subscription #6493

API to read event data without making a subscription #6493

Comments

Gerold103 commented Oct 5, 2021 • edited by locker

unera commented Feb 28, 2023

locker commented May 12, 2023

Gerold103 commented Oct 5, 2021 •

edited by locker