New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Develop quorum collection mechanism #4844

Closed

sergos opened this issue Apr 3, 2020 · 0 comments

Assignees

Labels

feature qsync replication

Milestone

Contributor

sergos commented Apr 3, 2020

Quorum (described at RFC here) should be collected per-lsn from a set of replicas in cluster that participate in the synchronous replication. The quorum considered as achieved for an lsn as soon as required number of replicas successfully reported bigger lsn in their vclocks.
As soon as replica reports an error, the replica should be disconnected as erroneous.
Upon quorum is achieved there should happen:

write quorum op in WAL
respond Ok to the customer's request

sergos mentioned this issue

Implement quorum-based synchronous replication #4842

Open

12 tasks

sergos added the qsync replication label

kyukhin added prio1 feature labels

kyukhin added this to the 2.5.1 milestone

Gerold103 self-assigned this

Gerold103 added a commit that referenced this issue


          replication: introduce space.is_sync option

c3f62c2

Synchronous space makes every transaction, affecting its data,
wait until it is replicated on a quorum of replicas before it is
committed.

Part of #4844

Gerold103 added a commit that referenced this issue


          replication: introduce replication_sync_quorum cfg

28e9a57

Synchronous transactions are supposed to be replicated on a
specified number of replicas before committed on master. The
number of replicas can be specified using replication_sync_quorum
option. It is 1 by default, so sync transactions work like
asynchronous when not configured anyhow. 1 means successful WAL
write on master is enough for commit.

Part of #4844

Gerold103 added a commit that referenced this issue


          txn: add TXN_WAIT_ACK flag

2ce7db9

When a transaction touches a synchronous space, its commit
procedure changes. The transaction enters state of 'waiting for
acks' from replicas and from own master's WAL.

To denote the state there is a new transaction flag -
TXN_WAIT_ACK.

Part of #4844

Gerold103 added a commit that referenced this issue


          replication: make sync transactions wait quorum

03d40b5

Synchronous transaction (which changes anything in a synchronous
space) before commit waits until it is replicated onto a quorum
of replicas.

So far all the 'synchronousness' is basically the same as the well
known 'wait_lsn' technique. With the exception, that the
transaction really is not committed until replicated.

Problem of wait_lsn is still present though, in case master
restarts. Because there is no a 'confirm' record in WAL telling
which transactions are replicated and can be applied.

Closes #4844
Closes #4845

sergepetrenko pushed a commit that referenced this issue


          replication: introduce space.is_sync option

2abfd57

Synchronous space makes every transaction, affecting its data,
wait until it is replicated on a quorum of replicas before it is
committed.

Part of #4844

sergepetrenko pushed a commit that referenced this issue


          replication: introduce replication_sync_quorum cfg

Synchronous transactions are supposed to be replicated on a
specified number of replicas before committed on master. The
number of replicas can be specified using replication_sync_quorum
option. It is 1 by default, so sync transactions work like
asynchronous when not configured anyhow. 1 means successful WAL
write on master is enough for commit.

Part of #4844

sergepetrenko pushed a commit that referenced this issue


          txn: add TXN_WAIT_ACK flag

8dfd1e1

When a transaction touches a synchronous space, its commit
procedure changes. The transaction enters state of 'waiting for
acks' from replicas and from own master's WAL.

To denote the state there is a new transaction flag -
TXN_WAIT_ACK.

Part of #4844

sergepetrenko pushed a commit that referenced this issue


          replication: make sync transactions wait quorum

d19b817

Synchronous transaction (which changes anything in a synchronous
space) before commit waits until it is replicated onto a quorum
of replicas.

So far all the 'synchronousness' is basically the same as the well
known 'wait_lsn' technique. With the exception, that the
transaction really is not committed until replicated.

Problem of wait_lsn is still present though, in case master
restarts. Because there is no a 'confirm' record in WAL telling
which transactions are replicated and can be applied.

Closes #4844
Closes #4845

sergepetrenko pushed a commit that referenced this issue


          replication: introduce space.is_sync option

4f18097

Synchronous space makes every transaction, affecting its data,
wait until it is replicated on a quorum of replicas before it is
committed.

Part of #4844

sergepetrenko pushed a commit that referenced this issue


          replication: introduce replication_sync_quorum cfg

04e1471

Synchronous transactions are supposed to be replicated on a
specified number of replicas before committed on master. The
number of replicas can be specified using replication_sync_quorum
option. It is 1 by default, so sync transactions work like
asynchronous when not configured anyhow. 1 means successful WAL
write on master is enough for commit.

Part of #4844

sergepetrenko pushed a commit that referenced this issue


          txn: add TXN_WAIT_ACK flag

e7a563a

When a transaction touches a synchronous space, its commit
procedure changes. The transaction enters state of 'waiting for
acks' from replicas and from own master's WAL.

To denote the state there is a new transaction flag -
TXN_WAIT_ACK.

Part of #4844

sergepetrenko pushed a commit that referenced this issue


          replication: make sync transactions wait quorum

8fd9112

Synchronous transaction (which changes anything in a synchronous
space) before commit waits until it is replicated onto a quorum
of replicas.

So far all the 'synchronousness' is basically the same as the well
known 'wait_lsn' technique. With the exception, that the
transaction really is not committed until replicated.

Problem of wait_lsn is still present though, in case master
restarts. Because there is no a 'confirm' record in WAL telling
which transactions are replicated and can be applied.

Closes #4844
Closes #4845

sergepetrenko pushed a commit that referenced this issue


          replication: introduce space.is_sync option

cb205a6

Synchronous space makes every transaction, affecting its data,
wait until it is replicated on a quorum of replicas before it is
committed.

Part of #4844

sergepetrenko pushed a commit that referenced this issue


          replication: introduce replication_sync_quorum cfg

3a3914f

Synchronous transactions are supposed to be replicated on a
specified number of replicas before committed on master. The
number of replicas can be specified using replication_sync_quorum
option. It is 1 by default, so sync transactions work like
asynchronous when not configured anyhow. 1 means successful WAL
write on master is enough for commit.

Part of #4844

sergepetrenko pushed a commit that referenced this issue


          txn: add TXN_WAIT_ACK flag

a20e51e

When a transaction touches a synchronous space, its commit
procedure changes. The transaction enters state of 'waiting for
acks' from replicas and from own master's WAL.

To denote the state there is a new transaction flag -
TXN_WAIT_ACK.

Part of #4844

sergepetrenko pushed a commit that referenced this issue


          replication: make sync transactions wait quorum

a0264c4

Synchronous transaction (which changes anything in a synchronous
space) before commit waits until it is replicated onto a quorum
of replicas.

So far all the 'synchronousness' is basically the same as the well
known 'wait_lsn' technique. With the exception, that the
transaction really is not committed until replicated.

Problem of wait_lsn is still present though, in case master
restarts. Because there is no a 'confirm' record in WAL telling
which transactions are replicated and can be applied.

Closes #4844
Closes #4845

sergepetrenko pushed a commit that referenced this issue


          replication: introduce space.is_sync option

f3c31fd

Synchronous space makes every transaction, affecting its data,
wait until it is replicated on a quorum of replicas before it is
committed.

Part of #4844

sergepetrenko pushed a commit that referenced this issue


          replication: introduce replication_sync_quorum cfg

f90a6cd

Synchronous transactions are supposed to be replicated on a
specified number of replicas before committed on master. The
number of replicas can be specified using replication_sync_quorum
option. It is 1 by default, so sync transactions work like
asynchronous when not configured anyhow. 1 means successful WAL
write on master is enough for commit.

Part of #4844

sergepetrenko pushed a commit that referenced this issue


          txn: add TXN_WAIT_ACK flag

e2f3d3e

When a transaction touches a synchronous space, its commit
procedure changes. The transaction enters state of 'waiting for
acks' from replicas and from own master's WAL.

To denote the state there is a new transaction flag -
TXN_WAIT_ACK.

Part of #4844

sergepetrenko pushed a commit that referenced this issue


          replication: make sync transactions wait quorum

8ac7212

Synchronous transaction (which changes anything in a synchronous
space) before commit waits until it is replicated onto a quorum
of replicas.

So far all the 'synchronousness' is basically the same as the well
known 'wait_lsn' technique. With the exception, that the
transaction really is not committed until replicated.

Problem of wait_lsn is still present though, in case master
restarts. Because there is no a 'confirm' record in WAL telling
which transactions are replicated and can be applied.

Closes #4844
Closes #4845

sergepetrenko pushed a commit that referenced this issue


          replication: introduce space.is_sync option

3992b9b

Synchronous space makes every transaction, affecting its data,
wait until it is replicated on a quorum of replicas before it is
committed.

Part of #4844

sergepetrenko pushed a commit that referenced this issue


          replication: introduce replication_sync_quorum cfg

7ddc4d3

Synchronous transactions are supposed to be replicated on a
specified number of replicas before committed on master. The
number of replicas can be specified using replication_sync_quorum
option. It is 1 by default, so sync transactions work like
asynchronous when not configured anyhow. 1 means successful WAL
write on master is enough for commit.

Part of #4844

sergepetrenko pushed a commit that referenced this issue


          txn: add TXN_WAIT_ACK flag

e85bfdf

When a transaction touches a synchronous space, its commit
procedure changes. The transaction enters state of 'waiting for
acks' from replicas and from own master's WAL.

To denote the state there is a new transaction flag -
TXN_WAIT_ACK.

Part of #4844

sergepetrenko pushed a commit that referenced this issue


          replication: make sync transactions wait quorum

e79a8da

Synchronous transaction (which changes anything in a synchronous
space) before commit waits until it is replicated onto a quorum
of replicas.

So far all the 'synchronousness' is basically the same as the well
known 'wait_lsn' technique. With the exception, that the
transaction really is not committed until replicated.

Problem of wait_lsn is still present though, in case master
restarts. Because there is no a 'confirm' record in WAL telling
which transactions are replicated and can be applied.

Closes #4844
Closes #4845

sergepetrenko pushed a commit that referenced this issue


          replication: introduce replication_synchro_* cfg options

50fdf1f

Synchronous transactions are supposed to be replicated on a
specified number of replicas before committed on master. The
number of replicas can be specified using
replication_synchro_quorum option. It is 1 by default, so sync
transactions work like asynchronous when not configured anyhow.
1 means successful WAL write on master is enough for commit.

When replication_synchro_quorum is greater than 1, an instance has to
wait for the specified number of replicas to  reply with success. If
enough replies aren't collected during replication_synchro_timeout,
the instance rolls back the tx in question.

Part of #4844
Part of #5073

sergepetrenko pushed a commit that referenced this issue


          txn: add TXN_WAIT_ACK flag

e7f34da

When a transaction touches a synchronous space, its commit
procedure changes. The transaction enters state of 'waiting for
acks' from replicas and from own master's WAL.

To denote the state there is a new transaction flag -
TXN_WAIT_ACK.

Part of #4844

sergepetrenko pushed a commit that referenced this issue


          replication: make sync transactions wait quorum

Synchronous transaction (which changes anything in a synchronous
space) before commit waits until it is replicated onto a quorum
of replicas.

So far all the 'synchronousness' is basically the same as the well
known 'wait_lsn' technique. With the exception, that the
transaction really is not committed until replicated.

Problem of wait_lsn is still present though, in case master
restarts. Because there is no a 'confirm' record in WAL telling
which transactions are replicated and can be applied.

Closes #4844
Closes #4845

sergepetrenko pushed a commit that referenced this issue


          replication: introduce space.is_sync option

b60dcc7

Synchronous space makes every transaction, affecting its data,
wait until it is replicated on a quorum of replicas before it is
committed.

Part of #4844
Part of #5073

sergepetrenko pushed a commit that referenced this issue


          replication: introduce replication_synchro_* cfg options

ff7c711

Synchronous transactions are supposed to be replicated on a
specified number of replicas before committed on master. The
number of replicas can be specified using
replication_synchro_quorum option. It is 1 by default, so sync
transactions work like asynchronous when not configured anyhow.
1 means successful WAL write on master is enough for commit.

When replication_synchro_quorum is greater than 1, an instance has to
wait for the specified number of replicas to  reply with success. If
enough replies aren't collected during replication_synchro_timeout,
the instance rolls back the tx in question.

Part of #4844
Part of #5073

sergepetrenko pushed a commit that referenced this issue


          txn: add TXN_WAIT_ACK flag

4148a7f

When a transaction touches a synchronous space, its commit
procedure changes. The transaction enters state of 'waiting for
acks' from replicas and from own master's WAL.

To denote the state there is a new transaction flag -
TXN_WAIT_ACK.

Part of #4844

sergepetrenko pushed a commit that referenced this issue


          replication: make sync transactions wait quorum

b621f64

Synchronous transaction (which changes anything in a synchronous
space) before commit waits until it is replicated onto a quorum
of replicas.

So far all the 'synchronousness' is basically the same as the well
known 'wait_lsn' technique. With the exception, that the
transaction really is not committed until replicated.

Problem of wait_lsn is still present though, in case master
restarts. Because there is no a 'confirm' record in WAL telling
which transactions are replicated and can be applied.

Closes #4844
Closes #4845

Gerold103 added a commit that referenced this issue


          replication: introduce space.is_sync option

Synchronous space makes every transaction, affecting its data,
wait until it is replicated on a quorum of replicas before it is
committed.

Part of #4844
Part of #5073

Gerold103 added a commit that referenced this issue


          replication: introduce replication_synchro_* cfg options

9ece5bf

Synchronous transactions are supposed to be replicated on a
specified number of replicas before committed on master. The
number of replicas can be specified using
replication_synchro_quorum option. It is 1 by default, so sync
transactions work like asynchronous when not configured anyhow.
1 means successful WAL write on master is enough for commit.

When replication_synchro_quorum is greater than 1, an instance has to
wait for the specified number of replicas to  reply with success. If
enough replies aren't collected during replication_synchro_timeout,
the instance rolls back the tx in question.

Part of #4844
Part of #5073

Gerold103 added a commit that referenced this issue


          txn: add TXN_WAIT_ACK flag

b31abf9

When a transaction touches a synchronous space, its commit
procedure changes. The transaction enters state of 'waiting for
acks' from replicas and from own master's WAL.

To denote the state there is a new transaction flag -
TXN_WAIT_ACK.

Part of #4844

Gerold103 added a commit that referenced this issue


          replication: make sync transactions wait quorum

38933b5

Synchronous transaction (which changes anything in a synchronous
space) before commit waits until it is replicated onto a quorum
of replicas.

So far all the 'synchronousness' is basically the same as the well
known 'wait_lsn' technique. With the exception, that the
transaction really is not committed until replicated.

Problem of wait_lsn is still present though, in case master
restarts. Because there is no a 'confirm' record in WAL telling
which transactions are replicated and can be applied.

Closes #4844
Closes #4845

Gerold103 added a commit that referenced this issue


          replication: introduce space.is_sync option

606d1f7

Synchronous space makes every transaction, affecting its data,
wait until it is replicated on a quorum of replicas before it is
committed.

Part of #4844
Part of #5073

Gerold103 added a commit that referenced this issue


          replication: introduce replication_synchro_* cfg options

980f7f3

Synchronous transactions are supposed to be replicated on a
specified number of replicas before committed on master. The
number of replicas can be specified using
replication_synchro_quorum option. It is 1 by default, so sync
transactions work like asynchronous when not configured anyhow.
1 means successful WAL write on master is enough for commit.

When replication_synchro_quorum is greater than 1, an instance has to
wait for the specified number of replicas to  reply with success. If
enough replies aren't collected during replication_synchro_timeout,
the instance rolls back the tx in question.

Part of #4844
Part of #5073

Gerold103 added a commit that referenced this issue


          txn: add TXN_WAIT_ACK flag

818e207

When a transaction touches a synchronous space, its commit
procedure changes. The transaction enters state of 'waiting for
acks' from replicas and from own master's WAL.

To denote the state there is a new transaction flag -
TXN_WAIT_ACK.

Part of #4844

Gerold103 added a commit that referenced this issue


          replication: make sync transactions wait quorum

dca9ae8

Synchronous transaction (which changes anything in a synchronous
space) before commit waits until it is replicated onto a quorum
of replicas.

So far all the 'synchronousness' is basically the same as the well
known 'wait_lsn' technique. With the exception, that the
transaction really is not committed until replicated.

Problem of wait_lsn is still present though, in case master
restarts. Because there is no a 'confirm' record in WAL telling
which transactions are replicated and can be applied.

Closes #4844
Closes #4845

Gerold103 added a commit that referenced this issue


          replication: introduce replication_synchro_* cfg options

4b79062

Synchronous transactions are supposed to be replicated on a
specified number of replicas before committed on master. The
number of replicas can be specified using
replication_synchro_quorum option. It is 1 by default, so sync
transactions work like asynchronous when not configured anyhow.
1 means successful WAL write on master is enough for commit.

When replication_synchro_quorum is greater than 1, an instance has to
wait for the specified number of replicas to reply with success. If
enough replies aren't collected during replication_synchro_timeout,
the instance rolls back the tx in question.

Part of #4844
Part of #5073

Gerold103 added a commit that referenced this issue


          replication: make sync transactions wait quorum

98dbe5e

Synchronous transaction (which changes anything in a synchronous
space) before commit waits until it is replicated onto a quorum
of replicas.

When there is a not committed synchronous transaction, any attempt
to commit a next transaction is suspended, even if it is an async
transaction.

This restriction comes from the theoretically possible dependency
of what is written in the async transactions on what was written
in the previous sync transactions.

So far all the 'synchronousness' is basically the same as the well
known 'wait_lsn' technique. With the exception, that the
transaction really is not committed until replicated.

Problem of wait_lsn is still present though, in case master
restarts. Because there is no a 'confirm' record in WAL telling
which transactions are replicated and can be applied.

Closes #4844
Closes #4845

Gerold103 added a commit that referenced this issue


          replication: introduce replication_synchro_* cfg options

5076e51

Synchronous transactions are supposed to be replicated on a
specified number of replicas before committed on master. The
number of replicas can be specified using
replication_synchro_quorum option. It is 1 by default, so sync
transactions work like asynchronous when not configured anyhow.
1 means successful WAL write on master is enough for commit.

When replication_synchro_quorum is greater than 1, an instance has to
wait for the specified number of replicas to reply with success. If
enough replies aren't collected during replication_synchro_timeout,
the instance rolls back the tx in question.

Part of #4844
Part of #5073

Gerold103 added a commit that referenced this issue


          replication: make sync transactions wait quorum

fa60ccb

Synchronous transaction (which changes anything in a synchronous
space) before commit waits until it is replicated onto a quorum
of replicas.

When there is a not committed synchronous transaction, any attempt
to commit a next transaction is suspended, even if it is an async
transaction.

This restriction comes from the theoretically possible dependency
of what is written in the async transactions on what was written
in the previous sync transactions.

So far all the 'synchronousness' is basically the same as the well
known 'wait_lsn' technique. With the exception, that the
transaction really is not committed until replicated.

Problem of wait_lsn is still present though, in case master
restarts. Because there is no a 'confirm' record in WAL telling
which transactions are replicated and can be applied.

Closes #4844
Closes #4845

sergepetrenko pushed a commit that referenced this issue


          replication: introduce space.is_sync option

d2944f5

Synchronous space makes every transaction, affecting its data,
wait until it is replicated on a quorum of replicas before it is
committed.

Part of #4844
Part of #5073

sergepetrenko pushed a commit that referenced this issue


          replication: introduce replication_synchro_* cfg options

dbc2781

Synchronous transactions are supposed to be replicated on a
specified number of replicas before committed on master. The
number of replicas can be specified using
replication_synchro_quorum option. It is 1 by default, so sync
transactions work like asynchronous when not configured anyhow.
1 means successful WAL write on master is enough for commit.

When replication_synchro_quorum is greater than 1, an instance has to
wait for the specified number of replicas to reply with success. If
enough replies aren't collected during replication_synchro_timeout,
the instance rolls back the tx in question.

Part of #4844
Part of #5073

sergepetrenko pushed a commit that referenced this issue


          replication: make sync transactions wait quorum

ab64a07

Synchronous transaction (which changes anything in a synchronous
space) before commit waits until it is replicated onto a quorum
of replicas.

When there is a not committed synchronous transaction, any attempt
to commit a next transaction is suspended, even if it is an async
transaction.

This restriction comes from the theoretically possible dependency
of what is written in the async transactions on what was written
in the previous sync transactions.

So far all the 'synchronousness' is basically the same as the well
known 'wait_lsn' technique. With the exception, that the
transaction really is not committed until replicated.

Problem of wait_lsn is still present though, in case master
restarts. Because there is no a 'confirm' record in WAL telling
which transactions are replicated and can be applied.

Closes #4844
Closes #4845

Gerold103 added a commit that referenced this issue


          replication: make sync transactions wait quorum

68c4460

Synchronous transaction (which changes anything in a synchronous
space) before commit waits until it is replicated onto a quorum
of replicas.

When there is a not committed synchronous transaction, any attempt
to commit a next transaction is suspended, even if it is an async
transaction.

This restriction comes from the theoretically possible dependency
of what is written in the async transactions on what was written
in the previous sync transactions.

So far all the 'synchronousness' is basically the same as the well
known 'wait_lsn' technique. With the exception, that the
transaction really is not committed until replicated.

Problem of wait_lsn is still present though, in case master
restarts. Because there is no a 'confirm' record in WAL telling
which transactions are replicated and can be applied.

Closes #4844
Closes #4845

Gerold103 added a commit that referenced this issue


          replication: make sync transactions wait quorum

42b87b5

Synchronous transaction (which changes anything in a synchronous
space) before commit waits until it is replicated onto a quorum
of replicas.

When there is a not committed synchronous transaction, any attempt
to commit a next transaction is suspended, even if it is an async
transaction.

This restriction comes from the theoretically possible dependency
of what is written in the async transactions on what was written
in the previous sync transactions.

So far all the 'synchronousness' is basically the same as the well
known 'wait_lsn' technique. With the exception, that the
transaction really is not committed until replicated.

Problem of wait_lsn is still present though, in case master
restarts. Because there is no a 'confirm' record in WAL telling
which transactions are replicated and can be applied.

Closes #4844
Closes #4845

Gerold103 added a commit that referenced this issue


          replication: make sync transactions wait quorum

389582d

Synchronous transaction (which changes anything in a synchronous
space) before commit waits until it is replicated onto a quorum
of replicas.

When there is a not committed synchronous transaction, any attempt
to commit a next transaction is suspended, even if it is an async
transaction.

This restriction comes from the theoretically possible dependency
of what is written in the async transactions on what was written
in the previous sync transactions.

So far all the 'synchronousness' is basically the same as the well
known 'wait_lsn' technique. With the exception, that the
transaction really is not committed until replicated.

Problem of wait_lsn is still present though, in case master
restarts. Because there is no a 'confirm' record in WAL telling
which transactions are replicated and can be applied.

Closes #4844
Closes #4845

Gerold103 added a commit that referenced this issue


          replication: introduce space.is_sync option

c14563f

Synchronous space makes every transaction, affecting its data,
wait until it is replicated on a quorum of replicas before it is
committed.

Part of #4844
Part of #5073

Gerold103 added a commit that referenced this issue


          replication: introduce replication_synchro_* cfg options

4cc49f3

Synchronous transactions are supposed to be replicated on a
specified number of replicas before committed on master. The
number of replicas can be specified using
replication_synchro_quorum option. It is 1 by default, so sync
transactions work like asynchronous when not configured anyhow.
1 means successful WAL write on master is enough for commit.

When replication_synchro_quorum is greater than 1, an instance has to
wait for the specified number of replicas to reply with success. If
enough replies aren't collected during replication_synchro_timeout,
the instance rolls back the tx in question.

Part of #4844
Part of #5073

Gerold103 added a commit that referenced this issue


          replication: make sync transactions wait quorum

4cb8490

Synchronous transaction (which changes anything in a synchronous
space) before commit waits until it is replicated onto a quorum
of replicas.

When there is a not committed synchronous transaction, any attempt
to commit a next transaction is suspended, even if it is an async
transaction.

This restriction comes from the theoretically possible dependency
of what is written in the async transactions on what was written
in the previous sync transactions.

So far all the 'synchronousness' is basically the same as the well
known 'wait_lsn' technique. With the exception, that the
transaction really is not committed until replicated.

Problem of wait_lsn is still present though, in case master
restarts. Because there is no a 'confirm' record in WAL telling
which transactions are replicated and can be applied.

Closes #4844
Closes #4845

kyukhin closed this as completed in

c927fce

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment