New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Stop caching global grants or disconnect after a global grant #2763
Comments
I've encountered this issue in our production setup. |
Actually, the spurious "access denied" issue due to cached privileges can happen even if privileges were granted long ago - in case a client connects as guest before recovery is complete. Here's a sequence of events leading to the problem:
Observed by @rtokarev. @kyukhin it might be worth scheduling this issue for 1.10. |
* Vladimir Davydov <notifications@github.com> [19/06/05 15:09]:
The fix is:
- link all sessions into a list
- update all cached privileges or disconnect the session on global
grant/revoke, drop user, enable/disable user.
…--
Konstantin Osipov, Moscow, Russia
|
This last error ``` [035] ... [035] disconnected_cnt [035] --- [035] -- 1 [035] +- 2 [035] ... [035] conn:close() [035] --- [035] ... [035] disconnected_cnt [035] --- [035] -- 2 [035] +- 3 [035] ... [035] test_run:cmd('stop server connecter') [035] --- [035] ``` Happens because net.box is able to connect to tarantool before it has finished bootstrap. When connecting, net.box tries to fetch schema executing a couple of selects, but fails to pass access check since grants aren't applied yet. This is described in detail in #2763 (comment) So, alter the test so that it tolerates multiple connection failures. Closes #4273
This last error ``` [035] ... [035] disconnected_cnt [035] --- [035] -- 1 [035] +- 2 [035] ... [035] conn:close() [035] --- [035] ... [035] disconnected_cnt [035] --- [035] -- 2 [035] +- 3 [035] ... [035] test_run:cmd('stop server connecter') [035] --- [035] ``` Happens because net.box is able to connect to tarantool before it has finished bootstrap. When connecting, net.box tries to fetch schema executing a couple of selects, but fails to pass access check since grants aren't applied yet. This is described in detail in #2763 (comment) So, alter the test so that it tolerates multiple connection failures. Closes #4273
This last error ``` [035] ... [035] disconnected_cnt [035] --- [035] -- 1 [035] +- 2 [035] ... [035] conn:close() [035] --- [035] ... [035] disconnected_cnt [035] --- [035] -- 2 [035] +- 3 [035] ... [035] test_run:cmd('stop server connecter') [035] --- [035] ``` Happens because net.box is able to connect to tarantool before it has finished bootstrap. When connecting, net.box tries to fetch schema executing a couple of selects, but fails to pass access check since grants aren't applied yet. This is described in detail in #2763 (comment) So, alter the test so that it tolerates multiple connection failures. Closes #4273 (cherry picked from commit 1a2addb)
This last error ``` [035] ... [035] disconnected_cnt [035] --- [035] -- 1 [035] +- 2 [035] ... [035] conn:close() [035] --- [035] ... [035] disconnected_cnt [035] --- [035] -- 2 [035] +- 3 [035] ... [035] test_run:cmd('stop server connecter') [035] --- [035] ``` Happens because net.box is able to connect to tarantool before it has finished bootstrap. When connecting, net.box tries to fetch schema executing a couple of selects, but fails to pass access check since grants aren't applied yet. This is described in detail in #2763 (comment) So, alter the test so that it tolerates multiple connection failures. Closes #4273 (cherry picked from commit 1a2addb)
I would rather keep users connected, and update credentials cache. But should anything special be done in case a user was doing replication, and its right to work with _cluster was revoked? Should the user be disconnected? |
Session has real user's credentials. The credentials have auth token to obtain object access privileges (to a space, to an index, to a function, etc), and a mask of global universe privileges. Object privileges are not cached. Each access to an object is a lookup in a tree of user's privileges. Universe privileges are cached. And before this patch when universe privs of a user were changed, it was not reflected in his sessions. Therefore, the sessions opened before priv change had outdated universe privs. Now each session subscribes on privileges update of the real user. That solves a couple of real life problems: - If a user managed to connect after box.cfg started listening port, but before access was granted, then he needed a reconnect; - Even if access was granted, a user may connect after box.cfg listen, but before access *is recovered* from _priv space. It was not possible to fix without a reconnect. And this problem affected replication. Closes #2763 Part of #4535 Part of #4536
Struct credentials is a cache of user's universal privileges. It is static and is never changed after creation. That is a problem. If a user privileges are updated, it is not reflected in his existing credentials caches. This patch reworks credentials API so as now this struct is not just a container for several numbers. It is an object with standard methods like create(), destroy(). A credentials object still is not updated together with its source user, but now at least the API allows to fix that. Next patch will add a trigger to struct credentials to catch user privilege changes and update the cache. Part of #2763
Credentials is a cache of user universal privileges. And that cache can become outdated in case user privs were changed after creation of the cache. The patch makes user notify all its credentials caches about privs change, via a trigger. Each credentials object now registers itself in user and gets all updates. That solves a couple of real life problems: - If a user managed to connect after box.cfg started listening port, but before access was granted, then he needed a reconnect; - Even if access was granted, a user may connect after box.cfg listen, but before access *is recovered* from _priv space. It was not possible to fix without a reconnect. And this problem affected replication. Closes #2763 Part of #4535 Part of #4536 @TarantoolBot document Title: User privileges update affects existing sessions and objects Previously if user privileges were updated (via `box.schema.user.grant/revoke`), it was not reflected in already existing sessions and objects like functions. Now it is. For example: ``` box.cfg{listen = 3313} box.schema.user.create('test_user', {password = '1'}) function test1() return 'success' end c = require('net.box').connect(box.cfg.listen, { user = 'test_user', password = '1' }) -- Error, no access for this connection. c:call('test1') box.schema.user.grant('test_user', 'execute', 'universe') -- Now works, even though access was granted after -- connection. c:call('test1') ``` A similar thing happens now with `box.session.su` and functions created via `box.schema.func.create` with `setuid` flag. In other words, now user privileges update is reflected everywhere immediately.
Struct credentials is a cache of user's universal privileges. It is static and is never changed after creation. That is a problem. If a user privileges are updated, it is not reflected in his existing credentials caches. This patch reworks credentials API so as now this struct is not just a container for several numbers. It is an object with standard methods like create(), destroy(). A credentials object still is not updated together with its source user, but now at least the API allows to fix that. Next patch will link all struct credentials of a user into a list via which the user will be able to keep the credentials up to date. Part of #2763
Credentials is a cache of user universal privileges. And that cache can become outdated in case user privs were changed after creation of the cache. The patch makes user update all its credentials caches with new privileges, via a list of all creds. That solves a couple of real life problems: - If a user managed to connect after box.cfg started listening port, but before access was granted, then he needed a reconnect; - Even if access was granted, a user may connect after box.cfg listen, but before access *is recovered* from _priv space. It was not possible to fix without a reconnect. And this problem affected replication. Closes #2763 Part of #4535 Part of #4536 @TarantoolBot document Title: User privileges update affects existing sessions and objects Previously if user privileges were updated (via `box.schema.user.grant/revoke`), it was not reflected in already existing sessions and objects like functions. Now it is. For example: ``` box.cfg{listen = 3313} box.schema.user.create('test_user', {password = '1'}) function test1() return 'success' end c = require('net.box').connect(box.cfg.listen, { user = 'test_user', password = '1' }) -- Error, no access for this connection. c:call('test1') box.schema.user.grant('test_user', 'execute', 'universe') -- Now works, even though access was granted after -- connection. c:call('test1') ``` A similar thing happens now with `box.session.su` and functions created via `box.schema.func.create` with `setuid` flag. In other words, now user privileges update is reflected everywhere immediately.
Struct credentials is a cache of user's universal privileges. It is static and is never changed after creation. That is a problem. If a user privileges are updated, it is not reflected in his existing credentials caches. This patch reworks credentials API so as now this struct is not just a container for several numbers. It is an object with standard methods like create(), destroy(). A credentials object still is not updated together with its source user, but now at least the API allows to fix that. Next patch will link all struct credentials of a user into a list via which the user will be able to keep the credentials up to date. Part of #2763
Credentials is a cache of user universal privileges. And that cache can become outdated in case user privs were changed after creation of the cache. The patch makes user update all its credentials caches with new privileges, via a list of all creds. That solves a couple of real life problems: - If a user managed to connect after box.cfg started listening port, but before access was granted, then he needed a reconnect; - Even if access was granted, a user may connect after box.cfg listen, but before access *is recovered* from _priv space. It was not possible to fix without a reconnect. And this problem affected replication. Closes #2763 Part of #4535 Part of #4536 @TarantoolBot document Title: User privileges update affects existing sessions and objects Previously if user privileges were updated (via `box.schema.user.grant/revoke`), it was not reflected in already existing sessions and objects like functions. Now it is. For example: ``` box.cfg{listen = 3313} box.schema.user.create('test_user', {password = '1'}) function test1() return 'success' end c = require('net.box').connect(box.cfg.listen, { user = 'test_user', password = '1' }) -- Error, no access for this connection. c:call('test1') box.schema.user.grant('test_user', 'execute', 'universe') -- Now works, even though access was granted after -- connection. c:call('test1') ``` A similar thing happens now with `box.session.su` and functions created via `box.schema.func.create` with `setuid` flag. In other words, now user privileges update is reflected everywhere immediately.
Struct credentials is a cache of user's universal privileges. It is static and is never changed after creation. That is a problem. If a user privileges are updated, it is not reflected in his existing credentials caches. This patch reworks credentials API so as now this struct is not just a container for several numbers. It is an object with standard methods like create(), destroy(). A credentials object still is not updated together with its source user, but now at least the API allows to fix that. Next patch will link all struct credentials of a user into a list via which the user will be able to keep the credentials up to date. Part of #2763 (cherry picked from commit a8c3ebd)
Credentials is a cache of user universal privileges. And that cache can become outdated in case user privs were changed after creation of the cache. The patch makes user update all its credentials caches with new privileges, via a list of all creds. That solves a couple of real life problems: - If a user managed to connect after box.cfg started listening port, but before access was granted, then he needed a reconnect; - Even if access was granted, a user may connect after box.cfg listen, but before access *is recovered* from _priv space. It was not possible to fix without a reconnect. And this problem affected replication. Closes #2763 Part of #4535 Part of #4536 @TarantoolBot document Title: User privileges update affects existing sessions and objects Previously if user privileges were updated (via `box.schema.user.grant/revoke`), it was not reflected in already existing sessions and objects like functions. Now it is. For example: ``` box.cfg{listen = 3313} box.schema.user.create('test_user', {password = '1'}) function test1() return 'success' end c = require('net.box').connect(box.cfg.listen, { user = 'test_user', password = '1' }) -- Error, no access for this connection. c:call('test1') box.schema.user.grant('test_user', 'execute', 'universe') -- Now works, even though access was granted after -- connection. c:call('test1') ``` A similar thing happens now with `box.session.su` and functions created via `box.schema.func.create` with `setuid` flag. In other words, now user privileges update is reflected everywhere immediately. (cherry picked from commit 06dbcec)
Struct credentials is a cache of user's universal privileges. It is static and is never changed after creation. That is a problem. If a user privileges are updated, it is not reflected in his existing credentials caches. This patch reworks credentials API so as now this struct is not just a container for several numbers. It is an object with standard methods like create(), destroy(). A credentials object still is not updated together with its source user, but now at least the API allows to fix that. Next patch will link all struct credentials of a user into a list via which the user will be able to keep the credentials up to date. Part of #2763 (cherry picked from commit 6f11f2a)
Credentials is a cache of user universal privileges. And that cache can become outdated in case user privs were changed after creation of the cache. The patch makes user update all its credentials caches with new privileges, via a list of all creds. That solves a couple of real life problems: - If a user managed to connect after box.cfg started listening port, but before access was granted, then he needed a reconnect; - Even if access was granted, a user may connect after box.cfg listen, but before access *is recovered* from _priv space. It was not possible to fix without a reconnect. And this problem affected replication. Closes #2763 Part of #4535 Part of #4536 @TarantoolBot document Title: User privileges update affects existing sessions and objects Previously if user privileges were updated (via `box.schema.user.grant/revoke`), it was not reflected in already existing sessions and objects like functions. Now it is. For example: ``` box.cfg{listen = 3313} box.schema.user.create('test_user', {password = '1'}) function test1() return 'success' end c = require('net.box').connect(box.cfg.listen, { user = 'test_user', password = '1' }) -- Error, no access for this connection. c:call('test1') box.schema.user.grant('test_user', 'execute', 'universe') -- Now works, even though access was granted after -- connection. c:call('test1') ``` A similar thing happens now with `box.session.su` and functions created via `box.schema.func.create` with `setuid` flag. In other words, now user privileges update is reflected everywhere immediately. (cherry picked from commit e5149e3)
Struct credentials is a cache of user's universal privileges. It is static and is never changed after creation. That is a problem. If a user privileges are updated, it is not reflected in his existing credentials caches. This patch reworks credentials API so as now this struct is not just a container for several numbers. It is an object with standard methods like create(), destroy(). A credentials object still is not updated together with its source user, but now at least the API allows to fix that. Next patch will link all struct credentials of a user into a list via which the user will be able to keep the credentials up to date. Part of #2763 (cherry picked from commit a8c3ebd)
Credentials is a cache of user universal privileges. And that cache can become outdated in case user privs were changed after creation of the cache. The patch makes user update all its credentials caches with new privileges, via a list of all creds. That solves a couple of real life problems: - If a user managed to connect after box.cfg started listening port, but before access was granted, then he needed a reconnect; - Even if access was granted, a user may connect after box.cfg listen, but before access *is recovered* from _priv space. It was not possible to fix without a reconnect. And this problem affected replication. Closes #2763 Part of #4535 Part of #4536 @TarantoolBot document Title: User privileges update affects existing sessions and objects Previously if user privileges were updated (via `box.schema.user.grant/revoke`), it was not reflected in already existing sessions and objects like functions. Now it is. For example: ``` box.cfg{listen = 3313} box.schema.user.create('test_user', {password = '1'}) function test1() return 'success' end c = require('net.box').connect(box.cfg.listen, { user = 'test_user', password = '1' }) -- Error, no access for this connection. c:call('test1') box.schema.user.grant('test_user', 'execute', 'universe') -- Now works, even though access was granted after -- connection. c:call('test1') ``` A similar thing happens now with `box.session.su` and functions created via `box.schema.func.create` with `setuid` flag. In other words, now user privileges update is reflected everywhere immediately. (cherry picked from commit 06dbcec)
Struct credentials is a cache of user's universal privileges. It is static and is never changed after creation. That is a problem. If a user privileges are updated, it is not reflected in his existing credentials caches. This patch reworks credentials API so as now this struct is not just a container for several numbers. It is an object with standard methods like create(), destroy(). A credentials object still is not updated together with its source user, but now at least the API allows to fix that. Next patch will link all struct credentials of a user into a list via which the user will be able to keep the credentials up to date. Part of #2763 (cherry picked from commit a8c3ebd) (cherry picked from commit 6b15dce)
Credentials is a cache of user universal privileges. And that cache can become outdated in case user privs were changed after creation of the cache. The patch makes user update all its credentials caches with new privileges, via a list of all creds. That solves a couple of real life problems: - If a user managed to connect after box.cfg started listening port, but before access was granted, then he needed a reconnect; - Even if access was granted, a user may connect after box.cfg listen, but before access *is recovered* from _priv space. It was not possible to fix without a reconnect. And this problem affected replication. Closes #2763 Part of #4535 Part of #4536 @TarantoolBot document Title: User privileges update affects existing sessions and objects Previously if user privileges were updated (via `box.schema.user.grant/revoke`), it was not reflected in already existing sessions and objects like functions. Now it is. For example: ``` box.cfg{listen = 3313} box.schema.user.create('test_user', {password = '1'}) function test1() return 'success' end c = require('net.box').connect(box.cfg.listen, { user = 'test_user', password = '1' }) -- Error, no access for this connection. c:call('test1') box.schema.user.grant('test_user', 'execute', 'universe') -- Now works, even though access was granted after -- connection. c:call('test1') ``` A similar thing happens now with `box.session.su` and functions created via `box.schema.func.create` with `setuid` flag. In other words, now user privileges update is reflected everywhere immediately. (cherry picked from commit 06dbcec) (cherry picked from commit 2b599c0)
Struct credentials is a cache of user's universal privileges. It is static and is never changed after creation. That is a problem. If a user privileges are updated, it is not reflected in his existing credentials caches. This patch reworks credentials API so as now this struct is not just a container for several numbers. It is an object with standard methods like create(), destroy(). A credentials object still is not updated together with its source user, but now at least the API allows to fix that. Next patch will link all struct credentials of a user into a list via which the user will be able to keep the credentials up to date. Part of #2763 (cherry picked from commit a8c3ebd) (cherry picked from commit 6f11f2a)
Credentials is a cache of user universal privileges. And that cache can become outdated in case user privs were changed after creation of the cache. The patch makes user update all its credentials caches with new privileges, via a list of all creds. That solves a couple of real life problems: - If a user managed to connect after box.cfg started listening port, but before access was granted, then he needed a reconnect; - Even if access was granted, a user may connect after box.cfg listen, but before access *is recovered* from _priv space. It was not possible to fix without a reconnect. And this problem affected replication. Closes #2763 Part of #4535 Part of #4536 @TarantoolBot document Title: User privileges update affects existing sessions and objects Previously if user privileges were updated (via `box.schema.user.grant/revoke`), it was not reflected in already existing sessions and objects like functions. Now it is. For example: ``` box.cfg{listen = 3313} box.schema.user.create('test_user', {password = '1'}) function test1() return 'success' end c = require('net.box').connect(box.cfg.listen, { user = 'test_user', password = '1' }) -- Error, no access for this connection. c:call('test1') box.schema.user.grant('test_user', 'execute', 'universe') -- Now works, even though access was granted after -- connection. c:call('test1') ``` A similar thing happens now with `box.session.su` and functions created via `box.schema.func.create` with `setuid` flag. In other words, now user privileges update is reflected everywhere immediately. (cherry picked from commit 06dbcec) (cherry picked from commit e5149e3)
Struct credentials is a cache of user's universal privileges. It is static and is never changed after creation. That is a problem. If a user privileges are updated, it is not reflected in his existing credentials caches. This patch reworks credentials API so as now this struct is not just a container for several numbers. It is an object with standard methods like create(), destroy(). A credentials object still is not updated together with its source user, but now at least the API allows to fix that. Next patch will link all struct credentials of a user into a list via which the user will be able to keep the credentials up to date. Part of #2763 (cherry picked from commit a8c3ebd)
Struct credentials is a cache of user's universal privileges. It is static and is never changed after creation. That is a problem. If a user privileges are updated, it is not reflected in his existing credentials caches. This patch reworks credentials API so as now this struct is not just a container for several numbers. It is an object with standard methods like create(), destroy(). A credentials object still is not updated together with its source user, but now at least the API allows to fix that. Next patch will link all struct credentials of a user into a list via which the user will be able to keep the credentials up to date. Part of #2763 (cherry picked from commit 6f11f2a) (cherry picked from commit 6465570)
Credentials is a cache of user universal privileges. And that cache can become outdated in case user privs were changed after creation of the cache. The patch makes user update all its credentials caches with new privileges, via a list of all creds. That solves a couple of real life problems: - If a user managed to connect after box.cfg started listening port, but before access was granted, then he needed a reconnect; - Even if access was granted, a user may connect after box.cfg listen, but before access *is recovered* from _priv space. It was not possible to fix without a reconnect. And this problem affected replication. Closes #2763 Part of #4535 Part of #4536 @TarantoolBot document Title: User privileges update affects existing sessions and objects Previously if user privileges were updated (via `box.schema.user.grant/revoke`), it was not reflected in already existing sessions and objects like functions. Now it is. For example: ``` box.cfg{listen = 3313} box.schema.user.create('test_user', {password = '1'}) function test1() return 'success' end c = require('net.box').connect(box.cfg.listen, { user = 'test_user', password = '1' }) -- Error, no access for this connection. c:call('test1') box.schema.user.grant('test_user', 'execute', 'universe') -- Now works, even though access was granted after -- connection. c:call('test1') ``` A similar thing happens now with `box.session.su` and functions created via `box.schema.func.create` with `setuid` flag. In other words, now user privileges update is reflected everywhere immediately. (cherry picked from commit e5149e3) (cherry picked from commit a2b1dc8)
The admin user has universal privileges before bootstrap or recovery are done. That allows to, for example, bootstrap from a remote master, because to do that the admin should be able to insert into system spaces, such as _priv. But after the patch on online credentials update was implemented (#2763, 48d00b0) the admin could loose its universal access if, for example, a role was granted to him before universal access was recovered. That happened by two reasons: - Any change in access rights, even in granted roles, led to rebuild of universal access; - Any change in access rights updated the universal access in all existing sessions, thanks to #2763. What happened: two tarantools were started. One of them master, granted 'replication' role to admin. Second node, slave, tried to bootstrap from the master. The slave created an admin session and started loading data. After it loaded 'grant replication role to admin' command, this nullified admin universal access everywhere, including this session. Next rows could not be applied. Closes #4606
The admin user has universal privileges before bootstrap or recovery are done. That allows to, for example, bootstrap from a remote master, because to do that the admin should be able to insert into system spaces, such as _priv. But after the patch on online credentials update was implemented (#2763, 48d00b0) the admin could loose its universal access if, for example, a role was granted to him before universal access was recovered. That happened by two reasons: - Any change in access rights, even in granted roles, led to rebuild of universal access; - Any change in access rights updated the universal access in all existing sessions, thanks to #2763. What happened: two tarantools were started. One of them master, granted 'replication' role to admin. Second node, slave, tried to bootstrap from the master. The slave created an admin session and started loading data. After it loaded 'grant replication role to admin' command, this nullified admin universal access everywhere, including this session. Next rows could not be applied. Closes #4606
The admin user has universal privileges before bootstrap or recovery are done. That allows to, for example, bootstrap from a remote master, because to do that the admin should be able to insert into system spaces, such as _priv. But after the patch on online credentials update was implemented (#2763, 48d00b0) the admin could loose its universal access if, for example, a role was granted to him before universal access was recovered. That happened by two reasons: - Any change in access rights, even in granted roles, led to rebuild of universal access; - Any change in access rights updated the universal access in all existing sessions, thanks to #2763. What happened: two tarantools were started. One of them master, granted 'replication' role to admin. Second node, slave, tried to bootstrap from the master. The slave created an admin session and started loading data. After it loaded 'grant replication role to admin' command, this nullified admin universal access everywhere, including this session. Next rows could not be applied. Closes #4606
The admin user has universal privileges before bootstrap or recovery are done. That allows to, for example, bootstrap from a remote master, because to do that the admin should be able to insert into system spaces, such as _priv. But after the patch on online credentials update was implemented (#2763, 48d00b0) the admin could loose its universal access if, for example, a role was granted to him before universal access was recovered. That happened by two reasons: - Any change in access rights, even in granted roles, led to rebuild of universal access; - Any change in access rights updated the universal access in all existing sessions, thanks to #2763. What happened: two tarantools were started. One of them master, granted 'replication' role to admin. Second node, slave, tried to bootstrap from the master. The slave created an admin session and started loading data. After it loaded 'grant replication role to admin' command, this nullified admin universal access everywhere, including this session. Next rows could not be applied. Closes #4606
The admin user has universal privileges before bootstrap or recovery are done. That allows to, for example, bootstrap from a remote master, because to do that the admin should be able to insert into system spaces, such as _priv. But after the patch on online credentials update was implemented (#2763, 48d00b0) the admin could loose its universal access if, for example, a role was granted to him before universal access was recovered. That happened by two reasons: - Any change in access rights, even in granted roles, led to rebuild of universal access; - Any change in access rights updated the universal access in all existing sessions, thanks to #2763. What happened: two tarantools were started. One of them master, granted 'replication' role to admin. Second node, slave, tried to bootstrap from the master. The slave created an admin session and started loading data. After it loaded 'grant replication role to admin' command, this nullified admin universal access everywhere, including this session. Next rows could not be applied. Closes #4606
The admin user has universal privileges before bootstrap or recovery are done. That allows to, for example, bootstrap from a remote master, because to do that the admin should be able to insert into system spaces, such as _priv. But after the patch on online credentials update was implemented (#2763, 48d00b0) the admin could loose its universal access if, for example, a role was granted to him before universal access was recovered. That happened by two reasons: - Any change in access rights, even in granted roles, led to rebuild of universal access; - Any change in access rights updated the universal access in all existing sessions, thanks to #2763. What happened: two tarantools were started. One of them master, granted 'replication' role to admin. Second node, slave, tried to bootstrap from the master. The slave created an admin session and started loading data. After it loaded 'grant replication role to admin' command, this nullified admin universal access everywhere, including this session. Next rows could not be applied. Closes #4606 (cherry picked from commit 95237ac)
The admin user has universal privileges before bootstrap or recovery are done. That allows to, for example, bootstrap from a remote master, because to do that the admin should be able to insert into system spaces, such as _priv. But after the patch on online credentials update was implemented (#2763, 48d00b0) the admin could loose its universal access if, for example, a role was granted to him before universal access was recovered. That happened by two reasons: - Any change in access rights, even in granted roles, led to rebuild of universal access; - Any change in access rights updated the universal access in all existing sessions, thanks to #2763. What happened: two tarantools were started. One of them master, granted 'replication' role to admin. Second node, slave, tried to bootstrap from the master. The slave created an admin session and started loading data. After it loaded 'grant replication role to admin' command, this nullified admin universal access everywhere, including this session. Next rows could not be applied. Closes #4606 (cherry picked from commit 95237ac)
The admin user has universal privileges before bootstrap or recovery are done. That allows to, for example, bootstrap from a remote master, because to do that the admin should be able to insert into system spaces, such as _priv. But after the patch on online credentials update was implemented (#2763, 48d00b0) the admin could loose its universal access if, for example, a role was granted to him before universal access was recovered. That happened by two reasons: - Any change in access rights, even in granted roles, led to rebuild of universal access; - Any change in access rights updated the universal access in all existing sessions, thanks to #2763. What happened: two tarantools were started. One of them master, granted 'replication' role to admin. Second node, slave, tried to bootstrap from the master. The slave created an admin session and started loading data. After it loaded 'grant replication role to admin' command, this nullified admin universal access everywhere, including this session. Next rows could not be applied. Closes #4606
The admin user has universal privileges before bootstrap or recovery are done. That allows to, for example, bootstrap from a remote master, because to do that the admin should be able to insert into system spaces, such as _priv. But after the patch on online credentials update was implemented (#2763, 48d00b0) the admin could loose its universal access if, for example, a role was granted to him before universal access was recovered. That happened by two reasons: - Any change in access rights, even in granted roles, led to rebuild of universal access; - Any change in access rights updated the universal access in all existing sessions, thanks to #2763. What happened: two tarantools were started. One of them master, granted 'replication' role to admin. Second node, slave, tried to bootstrap from the master. The slave created an admin session and started loading data. After it loaded 'grant replication role to admin' command, this nullified admin universal access everywhere, including this session. Next rows could not be applied. Closes #4606 (cherry picked from commit 95237ac)
Added for tests with issues: app/fiber.test.lua gh-5341 app-tap/debug.test.lua gh-5346 app-tap/http_client.test.lua gh-5346 app-tap/inspector.test.lua gh-5346 box/gh-2763-session-credentials-update.test.lua gh-5363 box/hash_collation.test.lua gh-5247 box/lua.test.lua gh-5351 box/net.box_connect_triggers_gh-2858.test.lua gh-5247 box/net.box_incompatible_index-gh-1729.test.lua gh-5360 box/net.box_on_schema_reload-gh-1904.test.lua gh-5354 box/protocol.test.lua gh-5247 box/update.test.lua gh-5247 box-tap/net.box.test.lua gh-5346 replication/autobootstrap.test.lua gh-4533 replication/autobootstrap_guest.test.lua gh-4533 replication/ddl.test.lua gh-5337 replication/gh-3160-misc-heartbeats-on-master-changes.test.lua gh-4940 replication/gh-3247-misc-iproto-sequence-value-not-replicated.test.lua.test.lua gh-5357 replication/gh-3637-misc-error-on-replica-auth-fail.test.lua gh-5343 replication/long_row_timeout.test.lua gh-4351 replication/on_replace.test.lua gh-5344, gh-5349 replication/prune.test.lua gh-5361 replication/qsync_advanced.test.lua gh-5340 replication/qsync_basic.test.lua gh-5355 replication/replicaset_ro_mostly.test.lua gh-5342 replication/wal_rw_stress.test.lua gh-5347 replication-py/multi.test.py gh-5362 sql/prepared.test.lua test gh-5359 sql-tap/selectG.test.lua gh-5350 vinyl/ddl.test.lua gh-5338 vinyl/gh-3395-read-prepared-uncommitted.test.lua gh-5197 vinyl/iterator.test.lua gh-5336 vinyl/write_iterator_rand.test.lua gh-5356 xlog/panic_on_wal_error.test.lua gh-5348
Added for tests with issues: app/fiber.test.lua gh-5341 app-tap/debug.test.lua gh-5346 app-tap/http_client.test.lua gh-5346 app-tap/inspector.test.lua gh-5346 box/gh-2763-session-credentials-update.test.lua gh-5363 box/hash_collation.test.lua gh-5247 box/lua.test.lua gh-5351 box/net.box_connect_triggers_gh-2858.test.lua gh-5247 box/net.box_incompatible_index-gh-1729.test.lua gh-5360 box/net.box_on_schema_reload-gh-1904.test.lua gh-5354 box/protocol.test.lua gh-5247 box/update.test.lua gh-5247 box-tap/net.box.test.lua gh-5346 replication/autobootstrap.test.lua gh-4533 replication/autobootstrap_guest.test.lua gh-4533 replication/ddl.test.lua gh-5337 replication/gh-3160-misc-heartbeats-on-master-changes.test.lua gh-4940 replication/gh-3247-misc-iproto-sequence-value-not-replicated.test.lua.test.lua gh-5357 replication/gh-3637-misc-error-on-replica-auth-fail.test.lua gh-5343 replication/long_row_timeout.test.lua gh-4351 replication/on_replace.test.lua gh-5344, gh-5349 replication/prune.test.lua gh-5361 replication/qsync_advanced.test.lua gh-5340 replication/qsync_basic.test.lua gh-5355 replication/replicaset_ro_mostly.test.lua gh-5342 replication/wal_rw_stress.test.lua gh-5347 replication-py/multi.test.py gh-5362 sql/prepared.test.lua test gh-5359 sql-tap/selectG.test.lua gh-5350 vinyl/ddl.test.lua gh-5338 vinyl/gh-3395-read-prepared-uncommitted.test.lua gh-5197 vinyl/iterator.test.lua gh-5336 vinyl/write_iterator_rand.test.lua gh-5356 xlog/panic_on_wal_error.test.lua gh-5348
Added for tests with issues: app/fiber.test.lua gh-5341 app-tap/debug.test.lua gh-5346 app-tap/http_client.test.lua gh-5346 app-tap/inspector.test.lua gh-5346 box/gh-2763-session-credentials-update.test.lua gh-5363 box/hash_collation.test.lua gh-5247 box/lua.test.lua gh-5351 box/net.box_connect_triggers_gh-2858.test.lua gh-5247 box/net.box_incompatible_index-gh-1729.test.lua gh-5360 box/net.box_on_schema_reload-gh-1904.test.lua gh-5354 box/protocol.test.lua gh-5247 box/update.test.lua gh-5247 box-tap/net.box.test.lua gh-5346 replication/autobootstrap.test.lua gh-4533 replication/autobootstrap_guest.test.lua gh-4533 replication/ddl.test.lua gh-5337 replication/gh-3160-misc-heartbeats-on-master-changes.test.lua gh-4940 replication/gh-3247-misc-iproto-sequence-value-not-replicated.test.lua.test.lua gh-5357 replication/gh-3637-misc-error-on-replica-auth-fail.test.lua gh-5343 replication/long_row_timeout.test.lua gh-4351 replication/on_replace.test.lua gh-5344, gh-5349 replication/prune.test.lua gh-5361 replication/qsync_advanced.test.lua gh-5340 replication/qsync_basic.test.lua gh-5355 replication/replicaset_ro_mostly.test.lua gh-5342 replication/wal_rw_stress.test.lua gh-5347 replication-py/multi.test.py gh-5362 sql/prepared.test.lua test gh-5359 sql-tap/selectG.test.lua gh-5350 vinyl/ddl.test.lua gh-5338 vinyl/gh-3395-read-prepared-uncommitted.test.lua gh-5197 vinyl/iterator.test.lua gh-5336 vinyl/write_iterator_rand.test.lua gh-5356 xlog/panic_on_wal_error.test.lua gh-5348 (cherry picked from commit 75ba744)
Added for tests with issues: app/fiber.test.lua gh-5341 app-tap/debug.test.lua gh-5346 app-tap/http_client.test.lua gh-5346 app-tap/inspector.test.lua gh-5346 box/gh-2763-session-credentials-update.test.lua gh-5363 box/hash_collation.test.lua gh-5247 box/lua.test.lua gh-5351 box/net.box_connect_triggers_gh-2858.test.lua gh-5247 box/net.box_incompatible_index-gh-1729.test.lua gh-5360 box/net.box_on_schema_reload-gh-1904.test.lua gh-5354 box/protocol.test.lua gh-5247 box/update.test.lua gh-5247 box-tap/net.box.test.lua gh-5346 replication/autobootstrap.test.lua gh-4533 replication/autobootstrap_guest.test.lua gh-4533 replication/ddl.test.lua gh-5337 replication/gh-3160-misc-heartbeats-on-master-changes.test.lua gh-4940 replication/gh-3247-misc-iproto-sequence-value-not-replicated.test.lua.test.lua gh-5357 replication/gh-3637-misc-error-on-replica-auth-fail.test.lua gh-5343 replication/long_row_timeout.test.lua gh-4351 replication/on_replace.test.lua gh-5344, gh-5349 replication/prune.test.lua gh-5361 replication/qsync_advanced.test.lua gh-5340 replication/qsync_basic.test.lua gh-5355 replication/replicaset_ro_mostly.test.lua gh-5342 replication/wal_rw_stress.test.lua gh-5347 replication-py/multi.test.py gh-5362 sql/prepared.test.lua test gh-5359 sql-tap/selectG.test.lua gh-5350 vinyl/ddl.test.lua gh-5338 vinyl/gh-3395-read-prepared-uncommitted.test.lua gh-5197 vinyl/iterator.test.lua gh-5336 vinyl/write_iterator_rand.test.lua gh-5356 xlog/panic_on_wal_error.test.lua gh-5348 (cherry picked from commit 75ba744)
Added for tests with issues: app/fiber.test.lua gh-5341 app-tap/debug.test.lua gh-5346 app-tap/http_client.test.lua gh-5346 app-tap/inspector.test.lua gh-5346 box/gh-2763-session-credentials-update.test.lua gh-5363 box/hash_collation.test.lua gh-5247 box/lua.test.lua gh-5351 box/net.box_connect_triggers_gh-2858.test.lua gh-5247 box/net.box_incompatible_index-gh-1729.test.lua gh-5360 box/net.box_on_schema_reload-gh-1904.test.lua gh-5354 box/protocol.test.lua gh-5247 box/update.test.lua gh-5247 box-tap/net.box.test.lua gh-5346 replication/autobootstrap.test.lua gh-4533 replication/autobootstrap_guest.test.lua gh-4533 replication/ddl.test.lua gh-5337 replication/gh-3160-misc-heartbeats-on-master-changes.test.lua gh-4940 replication/gh-3247-misc-iproto-sequence-value-not-replicated.test.lua.test.lua gh-5357 replication/gh-3637-misc-error-on-replica-auth-fail.test.lua gh-5343 replication/long_row_timeout.test.lua gh-4351 replication/on_replace.test.lua gh-5344, gh-5349 replication/prune.test.lua gh-5361 replication/qsync_advanced.test.lua gh-5340 replication/qsync_basic.test.lua gh-5355 replication/replicaset_ro_mostly.test.lua gh-5342 replication/wal_rw_stress.test.lua gh-5347 replication-py/multi.test.py gh-5362 sql/prepared.test.lua test gh-5359 sql-tap/selectG.test.lua gh-5350 vinyl/ddl.test.lua gh-5338 vinyl/gh-3395-read-prepared-uncommitted.test.lua gh-5197 vinyl/iterator.test.lua gh-5336 vinyl/write_iterator_rand.test.lua gh-5356 xlog/panic_on_wal_error.test.lua gh-5348 (cherry picked from commit 75ba744)
There is a race condition with an external application at bootstrap:
Possible solutions:
The problem is observed at Beeline and Mail.Ru
The text was updated successfully, but these errors were encountered: