Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

config: support anonymous replicas #9432

Closed
Totktonada opened this issue Nov 29, 2023 · 0 comments · Fixed by #9418
Closed

config: support anonymous replicas #9432

Totktonada opened this issue Nov 29, 2023 · 0 comments · Fixed by #9418
Assignees
Labels
3.0 Target is 3.0 and all newer release/master branches config

Comments

@Totktonada
Copy link
Member

Totktonada commented Nov 29, 2023

There is a couple of problems we need to resolve regarding anonymous replicas in the declarative configuration code.

  • An anonymous replica shouldn't attempt to persist its name in _cluster.
  • An anonymous replica shouldn't be used as an upstream for non-anonymous replica.
  • (I would also don't use anonymous replicas as upstreams for other anonymous replicas by default.)
  • An anonymous replica shouldn't be chosen as a boostrap leader in replication.failover: supervised mode.
  • Verify that a replicaset contains at least one non-anonymous instance.
  • Verify that an anonymous replica is not configured in read-write mode (using database.mode or <repicaset>.leader options).
  • Verify that an anonymous replica is not configured with replication.election_mode != off.
  • An anonymous replica should be able to be bootstrapped from a replicaset, where all the instances are in read-only mode (because it doesn't need a write to _cluster to join the replicaset).
@Totktonada Totktonada added 3.0 Target is 3.0 and all newer release/master branches config labels Nov 29, 2023
@Totktonada Totktonada self-assigned this Nov 29, 2023
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 5, 2023
The commit effectively enables support of anonymous replicas in the
declarative configuration. It has several caveats (see the changelog
entry), which will be resolved in the following commits of the patchset.

An attempt to persist an instance name of an anonymous replica can't be
successful, because it has no entry in `_cluster` system space. Such an
attempt leads to ER_INSTANCE_NAME_MISMATCH error.

This commit patches the configuration applying logic to skip attempt to
set `box.cfg({instance_name = <...>})` if the instance is configured as
an anonymous replica using `replication.anon: true` option.

Part of tarantool#9432

NO_DOC=replication.anon option is already documented in the scope of
       tarantool/doc#3851. The bugfix
       shouldn't affect the documentation pages much, however related
       constraints are summarized in a documentation request in the last
       commit of the series.
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 5, 2023
This commit effectively allows to set `replication.anon: true` without
specifying `replication.peers`.

Without filtering out anonymous replicas from the list of upstreams we
get an error regarding attempt to use an anonymous replica as an
upstream for a non-anonymous instance.

Also, anonymous replicas are excluded from autogenerated upstream list
for other anonymous replicas. It makes the list the same on all the
peers.

A user can configure a custom data flow using `replication.peers`
option.

Part of tarantool#9432

NO_DOC=The documentation request is in the last commit of the series.
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 5, 2023
Filter out anonymous replicas when choosing a bootstrap leader in
`replication.failover: supervised` mode. An anonymous replica can't be
in read-write mode, so it can't be a replicaset bootstrap leader.

Part of tarantool#9432

NO_DOC=It is bugfix. However, this detail is mentioned in the
       documentation request is in the last commit of the series just in
       case.
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 5, 2023
A replicaset that contains only anonymous replicas can't be
bootstrapped, because all the instances must in read-only mode.

Part of tarantool#9432

NO_DOC=The documentation request is in the last commit of the series.
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 5, 2023
This commit adds several checks that are specific for
`replication.failover` mode.

* `replication.failover: off`: an anonymous replica shouldn't be set to
  read-write mode using `database.mode` option.
* `replication.failover: manual`: an anonymous replica shouldn't be
  configured as a replicaset leader using `<replicaset>.leader` option.
* `replication.failover: election`: an anonymous replica can't be
  configured with `replication.election_mode` other than `off`.

This commit also adjusts default `replication.election_mode` to `off`
for an anonymous replica if it is part of a `replication.failover:
election` replicaset (the default for a non-anonymous instance is
`candidate`).

Part of tarantool#9432

NO_DOC=The documentation request is in the last commit of the series.
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 5, 2023
This commit allows to bootstrap an anonymous replica from a replicaset,
where all the instances are in read-only mode.

The reason of the change is that there are no technical reasons to
forbid this action. An anonymous replica is not registered in `_cluster`
system space, so it can join a replicaset even if there are no writable
instances.

Fixes tarantool#9432

@TarantoolBot document
Title: config: anonymous replica is now supported

`replication.anon: true` option is now working.

There are configuration constraints that are related to anonymous
replicas.

* A replicaset must contain at least one non-anonymous instance.
* An anonymous replica can't be configured as writable instance using
  `database.mode` or `<replicaset>.leader` options.
* An anonymous replica can't be configured with
  `replication.election_mode` equals to `candidate`, `voter` or `manual`
  (only `off` is allowed).

A few more nuances about anonymous replicas:

* Anonymous replicas are filtered out from default upstream list.
* A `replication.failover: election` replicaset can contain anonymous
  replicas, but `replication.election_mode` defaults to `off` for them
  (unlike non-anonymous instances, where the default is `candidate`).
* `replication.failover: supervised` skips anonymous replicas, when
  choosing a bootstrap leader.
* A anonymous replica can joined a replicaset, which has all the
  instances in read-only mode (ulike a non-anonymous instance).

See details in [1] and [2].

[1]: tarantool#9432
[2]: tarantool#9418
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 5, 2023
This commit allows to bootstrap an anonymous replica from a replicaset,
where all the instances are in read-only mode.

The reason of the change is that there are no technical reasons to
forbid this action. An anonymous replica is not registered in `_cluster`
system space, so it can join a replicaset even if there are no writable
instances.

Fixes tarantool#9432

@TarantoolBot document
Title: config: anonymous replica is now supported

`replication.anon: true` option is now working.

There are configuration constraints that are related to anonymous
replicas.

* A replicaset must contain at least one non-anonymous instance.
* An anonymous replica can't be configured as writable instance using
  `database.mode` or `<replicaset>.leader` options.
* An anonymous replica can't be configured with
  `replication.election_mode` equals to `candidate`, `voter` or `manual`
  (only `off` is allowed).

A few more nuances about anonymous replicas:

* Anonymous replicas are filtered out from default upstream list.
* A `replication.failover: election` replicaset can contain anonymous
  replicas, but `replication.election_mode` defaults to `off` for them
  (unlike non-anonymous instances, where the default is `candidate`).
* `replication.failover: supervised` skips anonymous replicas, when
  choosing a bootstrap leader.
* A anonymous replica can joined a replicaset, which has all the
  instances in read-only mode (unlike a non-anonymous instance).

See details in [1] and [2].

[1]: tarantool#9432
[2]: tarantool#9418
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 5, 2023
The commit effectively enables support of anonymous replicas in the
declarative configuration. It has several caveats (see the changelog
entry), which will be resolved in the following commits of the patchset.

An attempt to persist an instance name of an anonymous replica can't be
successful, because it has no entry in `_cluster` system space. Such an
attempt leads to ER_INSTANCE_NAME_MISMATCH error.

This commit patches the configuration applying logic to skip attempt to
set `box.cfg({instance_name = <...>})` if the instance is configured as
an anonymous replica using `replication.anon: true` option.

Part of tarantool#9432

NO_DOC=replication.anon option is already documented in the scope of
       tarantool/doc#3851. The bugfix
       shouldn't affect the documentation pages much, however related
       constraints are summarized in a documentation request in the last
       commit of the series.
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 5, 2023
This commit effectively allows to set `replication.anon: true` without
specifying `replication.peers`.

Without filtering out anonymous replicas from the list of upstreams we
get an error regarding attempt to use an anonymous replica as an
upstream for a non-anonymous instance.

Also, anonymous replicas are excluded from autogenerated upstream list
for other anonymous replicas. It makes the list the same on all the
peers.

A user can configure a custom data flow using `replication.peers`
option.

Part of tarantool#9432

NO_DOC=The documentation request is in the last commit of the series.
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 5, 2023
Filter out anonymous replicas when choosing a bootstrap leader in
`replication.failover: supervised` mode. An anonymous replica can't be
in read-write mode, so it can't be a replicaset bootstrap leader.

Part of tarantool#9432

NO_DOC=It is bugfix. However, this detail is mentioned in the
       documentation request is in the last commit of the series just in
       case.
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 5, 2023
A replicaset that contains only anonymous replicas can't be
bootstrapped, because all the instances must in read-only mode.

Part of tarantool#9432

NO_DOC=The documentation request is in the last commit of the series.
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 5, 2023
This commit adds several checks that are specific for
`replication.failover` mode.

* `replication.failover: off`: an anonymous replica shouldn't be set to
  read-write mode using `database.mode` option.
* `replication.failover: manual`: an anonymous replica shouldn't be
  configured as a replicaset leader using `<replicaset>.leader` option.
* `replication.failover: election`: an anonymous replica can't be
  configured with `replication.election_mode` other than `off`.

This commit also adjusts default `replication.election_mode` to `off`
for an anonymous replica if it is part of a `replication.failover:
election` replicaset (the default for a non-anonymous instance is
`candidate`).

Part of tarantool#9432

NO_DOC=The documentation request is in the last commit of the series.
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 5, 2023
This commit allows to bootstrap an anonymous replica from a replicaset,
where all the instances are in read-only mode.

The reason of the change is that there are no technical reasons to
forbid this action. An anonymous replica is not registered in `_cluster`
system space, so it can join a replicaset even if there are no writable
instances.

Fixes tarantool#9432

@TarantoolBot document
Title: config: anonymous replica is now supported

`replication.anon: true` option is now working.

There are configuration constraints that are related to anonymous
replicas.

* A replicaset must contain at least one non-anonymous instance.
* An anonymous replica can't be configured as writable instance using
  `database.mode` or `<replicaset>.leader` options.
* An anonymous replica can't be configured with
  `replication.election_mode` equals to `candidate`, `voter` or `manual`
  (only `off` is allowed).

A few more nuances about anonymous replicas:

* Anonymous replicas are filtered out from default upstream list.
* A `replication.failover: election` replicaset can contain anonymous
  replicas, but `replication.election_mode` defaults to `off` for them
  (unlike non-anonymous instances, where the default is `candidate`).
* `replication.failover: supervised` skips anonymous replicas, when
  choosing a bootstrap leader.
* A anonymous replica can joined a replicaset, which has all the
  instances in read-only mode (unlike a non-anonymous instance).

See details in [1] and [2].

[1]: tarantool#9432
[2]: tarantool#9418
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 6, 2023
The commit effectively enables support of anonymous replicas in the
declarative configuration. It has several caveats (see the changelog
entry), which will be resolved in the following commits of the patchset.

An attempt to persist an instance name of an anonymous replica can't be
successful, because it has no entry in `_cluster` system space. Such an
attempt leads to ER_INSTANCE_NAME_MISMATCH error.

This commit patches the configuration applying logic to skip attempt to
set `box.cfg({instance_name = <...>})` if the instance is configured as
an anonymous replica using `replication.anon: true` option.

Part of tarantool#9432

NO_DOC=replication.anon option is already documented in the scope of
       tarantool/doc#3851. The bugfix
       shouldn't affect the documentation pages much, however related
       constraints are summarized in a documentation request in the last
       commit of the series.
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 6, 2023
This commit effectively allows to set `replication.anon: true` without
specifying `replication.peers`.

Without filtering out anonymous replicas from the list of upstreams we
get an error regarding attempt to use an anonymous replica as an
upstream for a non-anonymous instance.

Also, anonymous replicas are excluded from autogenerated upstream list
for other anonymous replicas. It makes the list the same on all the
peers.

A user can configure a custom data flow using `replication.peers`
option.

Part of tarantool#9432

NO_DOC=The documentation request is in the last commit of the series.
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 6, 2023
Filter out anonymous replicas when choosing a bootstrap leader in
`replication.failover: supervised` mode. An anonymous replica can't be
in read-write mode, so it can't be a replicaset bootstrap leader.

Part of tarantool#9432

NO_DOC=It is bugfix. However, this detail is mentioned in the
       documentation request is in the last commit of the series just in
       case.
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 6, 2023
A replicaset that contains only anonymous replicas can't be
bootstrapped, because all the instances must in read-only mode.

Part of tarantool#9432

NO_DOC=The documentation request is in the last commit of the series.
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 6, 2023
This commit adds several checks that are specific for
`replication.failover` mode.

* `replication.failover: off`: an anonymous replica shouldn't be set to
  read-write mode using `database.mode` option.
* `replication.failover: manual`: an anonymous replica shouldn't be
  configured as a replicaset leader using `<replicaset>.leader` option.
* `replication.failover: election`: an anonymous replica can't be
  configured with `replication.election_mode` other than `off`.

This commit also adjusts default `replication.election_mode` to `off`
for an anonymous replica if it is part of a `replication.failover:
election` replicaset (the default for a non-anonymous instance is
`candidate`).

Part of tarantool#9432

NO_DOC=The documentation request is in the last commit of the series.
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 6, 2023
This commit allows to bootstrap an anonymous replica from a replicaset,
where all the instances are in read-only mode.

The reason of the change is that there are no technical reasons to
forbid this action. An anonymous replica is not registered in `_cluster`
system space, so it can join a replicaset even if there are no writable
instances.

Fixes tarantool#9432

@TarantoolBot document
Title: config: anonymous replica is now supported

`replication.anon: true` option is now working.

There are configuration constraints that are related to anonymous
replicas.

* A replicaset must contain at least one non-anonymous instance.
* An anonymous replica can't be configured as writable instance using
  `database.mode` or `<replicaset>.leader` options.
* An anonymous replica can't be configured with
  `replication.election_mode` equals to `candidate`, `voter` or `manual`
  (only `off` is allowed).

A few more nuances about anonymous replicas:

* Anonymous replicas are filtered out from default upstream list.
* A `replication.failover: election` replicaset can contain anonymous
  replicas, but `replication.election_mode` defaults to `off` for them
  (unlike non-anonymous instances, where the default is `candidate`).
* `replication.failover: supervised` skips anonymous replicas, when
  choosing a bootstrap leader.
* A anonymous replica can joined a replicaset, which has all the
  instances in read-only mode (unlike a non-anonymous instance).

See details in [1] and [2].

[1]: tarantool#9432
[2]: tarantool#9418
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 6, 2023
The commit effectively enables support of anonymous replicas in the
declarative configuration. It has several caveats (see the changelog
entry), which will be resolved in the following commits of the patchset.

An attempt to persist an instance name of an anonymous replica can't be
successful, because it has no entry in `_cluster` system space. Such an
attempt leads to ER_INSTANCE_NAME_MISMATCH error.

This commit patches the configuration applying logic to skip attempt to
set `box.cfg({instance_name = <...>})` if the instance is configured as
an anonymous replica using `replication.anon: true` option.

Part of tarantool#9432

NO_DOC=replication.anon option is already documented in the scope of
       tarantool/doc#3851. The bugfix
       shouldn't affect the documentation pages much, however related
       constraints are summarized in a documentation request in the last
       commit of the series.
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 6, 2023
This commit effectively allows to set `replication.anon: true` without
specifying `replication.peers`.

Without filtering out anonymous replicas from the list of upstreams we
get an error regarding attempt to use an anonymous replica as an
upstream for a non-anonymous instance.

Also, anonymous replicas are excluded from autogenerated upstream list
for other anonymous replicas. It makes the list the same on all the
peers.

A user can configure a custom data flow using `replication.peers`
option.

Part of tarantool#9432

NO_DOC=The documentation request is in the last commit of the series.
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 6, 2023
Filter out anonymous replicas when choosing a bootstrap leader in
`replication.failover: supervised` mode. An anonymous replica can't be
in read-write mode, so it can't be a replicaset bootstrap leader.

Part of tarantool#9432

NO_DOC=It is bugfix. However, this detail is mentioned in the
       documentation request is in the last commit of the series just in
       case.
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 6, 2023
A replicaset that contains only anonymous replicas can't be
bootstrapped, because all the instances must be in read-only mode.

Part of tarantool#9432

NO_DOC=The documentation request is in the last commit of the series.
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 6, 2023
This commit adds several checks that are specific for
`replication.failover` mode.

* `replication.failover: off`: an anonymous replica shouldn't be set to
  read-write mode using `database.mode` option.
* `replication.failover: manual`: an anonymous replica shouldn't be
  configured as a replicaset leader using `<replicaset>.leader` option.
* `replication.failover: election`: an anonymous replica can't be
  configured with `replication.election_mode` other than `off`.

This commit also adjusts default `replication.election_mode` to `off`
for an anonymous replica if it is part of a `replication.failover:
election` replicaset (the default for a non-anonymous instance is
`candidate`).

Part of tarantool#9432

NO_DOC=The documentation request is in the last commit of the series.
Totktonada added a commit to Totktonada/tarantool that referenced this issue Dec 6, 2023
This commit allows to bootstrap an anonymous replica from a replicaset,
where all the instances are in read-only mode.

The reason of the change is that there are no technical reasons to
forbid this action. An anonymous replica is not registered in `_cluster`
system space, so it can join a replicaset even if there are no writable
instances.

Fixes tarantool#9432

@TarantoolBot document
Title: config: anonymous replica is now supported

`replication.anon: true` option is now working.

There are configuration constraints that are related to anonymous
replicas.

* A replicaset must contain at least one non-anonymous instance.
* An anonymous replica can't be configured as writable instance using
  `database.mode` or `<replicaset>.leader` options.
* An anonymous replica can't be configured with
  `replication.election_mode` equals to `candidate`, `voter` or `manual`
  (only `off` is allowed).

A few more nuances about anonymous replicas:

* Anonymous replicas are filtered out from default upstream list.
* A `replication.failover: election` replicaset can contain anonymous
  replicas, but `replication.election_mode` defaults to `off` for them
  (unlike non-anonymous instances, where the default is `candidate`).
* `replication.failover: supervised` skips anonymous replicas, when
  choosing a bootstrap leader.
* A anonymous replica can joined a replicaset, which has all the
  instances in read-only mode (unlike a non-anonymous instance).

See details in [1] and [2].

[1]: tarantool#9432
[2]: tarantool#9418
Totktonada added a commit that referenced this issue Dec 6, 2023
The commit effectively enables support of anonymous replicas in the
declarative configuration. It has several caveats (see the changelog
entry), which will be resolved in the following commits of the patchset.

An attempt to persist an instance name of an anonymous replica can't be
successful, because it has no entry in `_cluster` system space. Such an
attempt leads to ER_INSTANCE_NAME_MISMATCH error.

This commit patches the configuration applying logic to skip attempt to
set `box.cfg({instance_name = <...>})` if the instance is configured as
an anonymous replica using `replication.anon: true` option.

Part of #9432

NO_DOC=replication.anon option is already documented in the scope of
       tarantool/doc#3851. The bugfix
       shouldn't affect the documentation pages much, however related
       constraints are summarized in a documentation request in the last
       commit of the series.
Totktonada added a commit that referenced this issue Dec 6, 2023
This commit effectively allows to set `replication.anon: true` without
specifying `replication.peers`.

Without filtering out anonymous replicas from the list of upstreams we
get an error regarding attempt to use an anonymous replica as an
upstream for a non-anonymous instance.

Also, anonymous replicas are excluded from autogenerated upstream list
for other anonymous replicas. It makes the list the same on all the
peers.

A user can configure a custom data flow using `replication.peers`
option.

Part of #9432

NO_DOC=The documentation request is in the last commit of the series.
Totktonada added a commit that referenced this issue Dec 6, 2023
Filter out anonymous replicas when choosing a bootstrap leader in
`replication.failover: supervised` mode. An anonymous replica can't be
in read-write mode, so it can't be a replicaset bootstrap leader.

Part of #9432

NO_DOC=It is bugfix. However, this detail is mentioned in the
       documentation request is in the last commit of the series just in
       case.
Totktonada added a commit that referenced this issue Dec 6, 2023
A replicaset that contains only anonymous replicas can't be
bootstrapped, because all the instances must be in read-only mode.

Part of #9432

NO_DOC=The documentation request is in the last commit of the series.
Totktonada added a commit that referenced this issue Dec 6, 2023
This commit adds several checks that are specific for
`replication.failover` mode.

* `replication.failover: off`: an anonymous replica shouldn't be set to
  read-write mode using `database.mode` option.
* `replication.failover: manual`: an anonymous replica shouldn't be
  configured as a replicaset leader using `<replicaset>.leader` option.
* `replication.failover: election`: an anonymous replica can't be
  configured with `replication.election_mode` other than `off`.

This commit also adjusts default `replication.election_mode` to `off`
for an anonymous replica if it is part of a `replication.failover:
election` replicaset (the default for a non-anonymous instance is
`candidate`).

Part of #9432

NO_DOC=The documentation request is in the last commit of the series.
Totktonada added a commit that referenced this issue Dec 6, 2023
This commit allows to bootstrap an anonymous replica from a replicaset,
where all the instances are in read-only mode.

The reason of the change is that there are no technical reasons to
forbid this action. An anonymous replica is not registered in `_cluster`
system space, so it can join a replicaset even if there are no writable
instances.

Fixes #9432

@TarantoolBot document
Title: config: anonymous replica is now supported

`replication.anon: true` option is now working.

There are configuration constraints that are related to anonymous
replicas.

* A replicaset must contain at least one non-anonymous instance.
* An anonymous replica can't be configured as writable instance using
  `database.mode` or `<replicaset>.leader` options.
* An anonymous replica can't be configured with
  `replication.election_mode` equals to `candidate`, `voter` or `manual`
  (only `off` is allowed).

A few more nuances about anonymous replicas:

* Anonymous replicas are filtered out from default upstream list.
* A `replication.failover: election` replicaset can contain anonymous
  replicas, but `replication.election_mode` defaults to `off` for them
  (unlike non-anonymous instances, where the default is `candidate`).
* `replication.failover: supervised` skips anonymous replicas, when
  choosing a bootstrap leader.
* A anonymous replica can joined a replicaset, which has all the
  instances in read-only mode (unlike a non-anonymous instance).

See details in [1] and [2].

[1]: #9432
[2]: #9418
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.0 Target is 3.0 and all newer release/master branches config
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant