[docdb] Support Savepoints with xcluster transactional replication #14308
Assignees
Labels
area/docdb
YugabyteDB core features
kind/enhancement
This is an enhancement of an existing feature
priority/medium
Medium priority issue
xCluster
Label for xCluster related issues/improvements
Comments
shishir2001-yb
added
kind/bug
yugabyte-ci
added
kind/enhancement
lingamsandeep
changed the title
lingamsandeep
changed the title
lingamsandeep
changed the title
mdbridge
added a commit
that referenced
this issue
Apr 5, 2023
…pted when disabled by --enable_pg_savepoint Summary: When this flag is set to false, use of SAVEPOINT related features in SQL produces an error like: ``` 2023-03-15 23:47:51.117 UTC [722541] ERROR: SAVEPOINT <transaction> not supported yet at character 1 2023-03-15 23:47:51.117 UTC [722541] HINT: See #1125. React with thumbs up to raise its priority ``` This is misleading because it implies the entire feature is not yet supported whereas it's just unsupported because of gflag setting. Also, the issue pointed to above (1125) -- #1125 -- seems to be out of date because it claims that many implemented features are still unimplemented. This diff changes the error message to: ``` 2023-03-15 23:32:20.676 UTC [680415] ERROR: SAVEPOINT <transaction> not supported due to setting of flag --enable_pg_savepoints at character 1 ``` 3/29/2023 UPDATE: error message is now: ``` ERROR: SAVEPOINT <transaction> not supported due to setting of flag --enable_pg_savepoints LINE 1: SAVEPOINT A ^ HINT: flag may have set to false because savepoints do not currently work with xCluster replication (see #14308) ``` Fixes #16374 Test Plan: ``` yb_build.sh --sj --cxx-test pgwrapper_pg_mini-test --gtest_filter '*TestSavePointCanBeDisabled*' ``` Look at error message in test log. 3/29/2023 UPDATE: replace that test with one that checks the error message: ``` yb_build.sh --java-test org.yb.pgsql.TestPgSavepoints ``` Reviewers: slingam, jhe, rsami, hsunder Reviewed By: hsunder Subscribers: ybase, rsami Differential Revision: https://phabricator.dev.yugabyte.com/D23625
premkumr
pushed a commit
to premkumr/yugabyte-db
that referenced
this issue
Apr 10, 2023
…is attempted when disabled by --enable_pg_savepoint Summary: When this flag is set to false, use of SAVEPOINT related features in SQL produces an error like: ``` 2023-03-15 23:47:51.117 UTC [722541] ERROR: SAVEPOINT <transaction> not supported yet at character 1 2023-03-15 23:47:51.117 UTC [722541] HINT: See yugabyte#1125. React with thumbs up to raise its priority ``` This is misleading because it implies the entire feature is not yet supported whereas it's just unsupported because of gflag setting. Also, the issue pointed to above (1125) -- yugabyte#1125 -- seems to be out of date because it claims that many implemented features are still unimplemented. This diff changes the error message to: ``` 2023-03-15 23:32:20.676 UTC [680415] ERROR: SAVEPOINT <transaction> not supported due to setting of flag --enable_pg_savepoints at character 1 ``` 3/29/2023 UPDATE: error message is now: ``` ERROR: SAVEPOINT <transaction> not supported due to setting of flag --enable_pg_savepoints LINE 1: SAVEPOINT A ^ HINT: flag may have set to false because savepoints do not currently work with xCluster replication (see yugabyte#14308) ``` Fixes yugabyte#16374 Test Plan: ``` yb_build.sh --sj --cxx-test pgwrapper_pg_mini-test --gtest_filter '*TestSavePointCanBeDisabled*' ``` Look at error message in test log. 3/29/2023 UPDATE: replace that test with one that checks the error message: ``` yb_build.sh --java-test org.yb.pgsql.TestPgSavepoints ``` Reviewers: slingam, jhe, rsami, hsunder Reviewed By: hsunder Subscribers: ybase, rsami Differential Revision: https://phabricator.dev.yugabyte.com/D23625
mdbridge
added a commit
that referenced
this issue
Apr 22, 2023
…ction info
Summary:
Currently when we use RocksDB logging we get output like:
```
[P-ts-1] I0421 10:09:27.527050 14095 tablet.cc:1490] T 6797611b07a14e0e83d6241d15baf104 P 44f79f51d3f841f1bede12501d097d83: Wrote 5 key/value pairs to kIntents RocksDB:
...
T 6797611b07a14e0e83d6241d15baf104 P 44f79f51d3f841f1bede12501d097d83 [I]: 4. PutCF: SubDocKey(DocKey([], []), []) [kWeakRead, kWeakWrite] HT{ physical: 1682096967524220 w: 1 } => 'xa\xd6\x9f\x11&\x95Ds\x94\x86=\xdd\x1b\x95\x1c?n\x00\x00\x00\x02' (7861D69F112695447394863DDD1B951C3F6E00000002) (error: Corruption (yb/util/slice.cc:137): Wrong first byte, expected 119 but found 110)
```
The corruption error is because the code doesn't understand the subtransaction field.
After the fix in this diff, we get the correct:
```
[P-ts-1] I0421 10:04:30.982218 9659 tablet.cc:1490] T 0a2375b68b264664ba8681b1b9a58c31 P 16429418e9bf4173a80f63b2d15cae67: Wrote 5 key/value pairs to kIntents RocksDB:
...
T 0a2375b68b264664ba8681b1b9a58c31 P 16429418e9bf4173a80f63b2d15cae67 [I]: 4. PutCF: SubDocKey(DocKey([], []), []) [kWeakRead, kWeakWrite] HT{ physical: 1682096670973032 w: 1 } => TransactionId(72a7a781-bb69-4fc6-8e51-d4714db4dce0) SubTransactionId(2) none
```
Test Plan:
```
yb_build.sh --cxx-test xcluster_ysql-test --gtest_filter '*TransactionWithSavepoints' --test-args '--gtest_also_run_disabled_tests'
```
Reviewers: xCluster, jhe, azhan, sergei, hsunder
Reviewed By: hsunder
Subscribers: ybase
Differential Revision: https://phabricator.dev.yugabyte.com/D24688
mdbridge
added a commit
that referenced
this issue
May 4, 2023
…ter replication Summary: Added a test per the title. There are actually two variants of the test because there is an optimization pathway (see the code for tests) that is different enough that I feel that we need a separate test for it being used and a separate one for it not being used. The new tests are disabled in this diff because the underlying feature they are testing for is currently incompletely implemented, causing them to fail. The next diff in the stack fixes this and thus un-disables them. Jira Issue(s): DB-3745 Test Plan: ``` yb_build.sh --cxx-test xcluster_ysql-test --gtest_filter '*TransactionWithSavepoints*' ``` Reviewers: xCluster, hsunder, rahuldesirazu, jhe Reviewed By: rahuldesirazu, jhe Subscribers: rahuldesirazu, jhe, ybase Differential Revision: https://phabricator.dev.yugabyte.com/D24377
mdbridge
added a commit
that referenced
this issue
May 15, 2023
…AVEPOINTs
Summary:
Background:
A savepoint is a special mark inside a transaction that allows all commands that are executed after it was established to be rolled back, restoring the transaction state to what it was at the time of the savepoint.
In Yugabyte, Savepoints are implemented using sub-transactions, which are either committed or rolled back depending on whether user rolls back or releases the savepoint.
Each intent belonging to a transaction is labeled with a subtransaction ID. As the transaction proceeds, the transaction manager keeps track of which subtransactions have been aborted so far. This set is incorporated in the COMMITTED Raft record when the transaction is committed. This information is then used by the apply transaction logic to figure out which intents should actually be applied.
Before this diff, the subtransaction information, both for intents and for what was aborted, was not replicated across to the consumer side or used to do transaction applications.
What this diff does at a high level:
* in the CDC producer:
* propagates the set of aborted subtransactions as a part of the APPLY CDC record
* propagates with each intent what subtransaction it belongs to
* on the receiving xCluster side:
* place the what subtransactions were aborted in the apply external transactions info of the WriteRequestPB
* likewise, encode the subtransaction ID of each intent
* on the consumer tablet:
* using the received information, apply only the external intents belonging to subtransactions that have not been aborted
* add a new optional field of the compressed external intents format to store the subtransaction ID:
```
// External intents are stored in the following format:
// key: kExternalTransactionId, txn_id
// value: kUuid involved_tablet [kSubTransactionId subtransaction_ID]
// kExternalIntents size(intent1_key), intent1_key, size(intent1_value), intent1_value,
// size(intent2_key)... 0
// where size is encoded as varint.
* we omit the subtransaction ID here and elsewhere when it is 1, the only subtransaction ID used in normal transactions that do not need multiple subtransactions
```
Notes on the new compressed intent format:
* it supports rolling forward existing databases -- we still accept and use the original format
* it will *not* directly support version downgrades as the old code will not understand the new format
* intents are transient; likely it would be possible to create a run book to support rolling back
* we use no extra space unless a transaction actually has multiple sub transactions
* note that transactions using the default Postgres isolation level (READ COMMITTED RSN) will be using subtransactions if statements need to be retried
* currently a fixed four bytes are being used for the subtransaction ID field plus a one byte prefix; probably a variable length encoding could save three bytes here if we thought it was needed
What this diff does not do:
* make SAVEPOINT's work with the pre-Hari scheme xCluster transactional replication
Test Plan:
This diff enables the existing two failing tests for SAVEPOINT with transactional xCluster replication; they are now passing:
```
yb_build.sh --cxx-test xcluster_ysql-test --gtest_filter '*TransactionWithSavepoints*'
```
For better debugging messages right now, turn off packing:
```
yb_build.sh --cxx-test xcluster_ysql-test --gtest_filter '*TransactionWithSavepoints*' --test-args '--ysql_enable_packed_row=false'
```
I think you get the clearest sense of what's going on by looking at the intents of one producer TServer and one consumer TServer.
For the test without optimization (TransactionWithSavepointsNoOpt), the SQL we are running is:
```
// Create two SAVEPOINTs but abort only one of them; all the writes except the aborted one should
// be replicated and visible on the consumer side.
ASSERT_OK(conn.Execute("BEGIN"));
ASSERT_OK(conn.ExecuteFormat("insert into $0 values(1777777)", table_name));
ASSERT_OK(conn.Execute("SAVEPOINT a"));
ASSERT_OK(conn.ExecuteFormat("insert into $0 values(2777777)", table_name));
ASSERT_OK(conn.Execute("RELEASE SAVEPOINT a"));
ASSERT_OK(conn.Execute("SAVEPOINT b"));
ASSERT_OK(conn.ExecuteFormat("insert into $0 values(3777777)", table_name));
ASSERT_OK(conn.Execute("ROLLBACK TO SAVEPOINT b"));
ASSERT_OK(conn.ExecuteFormat("insert into $0 values(4777777)", table_name));
ASSERT_OK(conn.Execute("COMMIT"));
```
That results in the following (omitting uninteresting writes) on the producer tablet:
```
# intents in the uncompressed format for [1-4]777777, with corresponding subtransaction IDs
# remember that we omit sub transaction ID 1.
[P-ts-1] I0424 15:10:25.392364 2390 tablet.cc:1502] T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35: Wrote 7 key/value pairs to kIntents RocksDB:
[P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [I]: 1. PutCF: TXN META eed9b150-0180-498c-bee9-eb2072b3f12d => { transaction_id: eed9b150-0180-498c-bee9-eb2072b3f12d isolation: SNAPSHOT_ISOLATION status_tablet: a13645c266084622bfe6e29190338041 priority: 15920732136751982613 start_time: { days: 19471 time: 15:10:25.380609 } locality: GLOBAL old_status_tablet: external_transaction: 0}
[P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [I]: 2. PutCF: SubDocKey(DocKey(0x56f1, [1777777], []), [SystemColumnId(0)]) [kStrongRead, kStrongWrite] HT{ days: 19471 time: 15:10:25.388093 } => TransactionId(eed9b150-0180-498c-bee9-eb2072b3f12d) WriteId(0) null
[P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [I]: 3. PutCF: TXN REV eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.388093 } => SubDocKey(DocKey(0x56f1, [1777777], []), [SystemColumnId(0)]) [kStrongRead, kStrongWrite] HT{ days: 19471 time: 15:10:25.388093 }
[P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [I]: 4. PutCF: SubDocKey(DocKey(0x56f1, [1777777], []), []) [kWeakRead, kWeakWrite] HT{ days: 19471 time: 15:10:25.388093 w: 1 } => TransactionId(eed9b150-0180-498c-bee9-eb2072b3f12d) none
[P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [I]: 5. PutCF: TXN REV eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.388093 w: 1 } => SubDocKey(DocKey(0x56f1, [1777777], []), []) [kWeakRead, kWeakWrite] HT{ days: 19471 time: 15:10:25.388093 w: 1 }
[P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [I]: 6. PutCF: SubDocKey(DocKey([], []), []) [kWeakRead, kWeakWrite] HT{ days: 19471 time: 15:10:25.388093 w: 2 } => TransactionId(eed9b150-0180-498c-bee9-eb2072b3f12d) none
[P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [I]: 7. PutCF: TXN REV eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.388093 w: 2 } => SubDocKey(DocKey([], []), []) [kWeakRead, kWeakWrite] HT{ days: 19471 time: 15:10:25.388093 w: 2 }
[P-ts-1] I0424 15:10:25.396523 2845 tablet.cc:1502] T 77394a9a5e254a619ec409feef132a4a P c630698bd61844ed8dd059f9b9b30d35: Wrote 7 key/value pairs to kIntents RocksDB:
[P-ts-1]: T 77394a9a5e254a619ec409feef132a4a P c630698bd61844ed8dd059f9b9b30d35 [I]: 2. PutCF: SubDocKey(DocKey(0xdaa5, [2777777], []), [SystemColumnId(0)]) [kStrongRead, kStrongWrite] HT{ days: 19471 time: 15:10:25.393740 } => TransactionId(eed9b150-0180-498c-bee9-eb2072b3f12d) SubTransactionId(2) WriteId(0) null
[P-ts-1] I0424 15:10:25.403523 2390 tablet.cc:1502] T a380379d15f84ea39791f18cf5c20f3a P c630698bd61844ed8dd059f9b9b30d35: Wrote 7 key/value pairs to kIntents RocksDB:
[P-ts-1]: T a380379d15f84ea39791f18cf5c20f3a P c630698bd61844ed8dd059f9b9b30d35 [I]: 2. PutCF: SubDocKey(DocKey(0x2ead, [3777777], []), [SystemColumnId(0)]) [kStrongRead, kStrongWrite] HT{ days: 19471 time: 15:10:25.399436 } => TransactionId(eed9b150-0180-498c-bee9-eb2072b3f12d) SubTransactionId(3) WriteId(0) null
[P-ts-1] I0424 15:10:25.418630 2349 tablet.cc:1502] T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35: Wrote 6 key/value pairs to kIntents RocksDB:
[P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [I]: 1. PutCF: SubDocKey(DocKey(0x58db, [4777777], []), [SystemColumnId(0)]) [kStrongRead, kStrongWrite] HT{ days: 19471 time: 15:10:25.411547 } => TransactionId(eed9b150-0180-498c-bee9-eb2072b3f12d) SubTransactionId(4) WriteId(1) null
# start application: we write all of these except 3777777 and delete all the intents
[P-ts-1] I0424 15:10:25.544713 2845 tablet.cc:1502] T 77394a9a5e254a619ec409feef132a4a P c630698bd61844ed8dd059f9b9b30d35: Wrote 1 key/value pairs to kRegular RocksDB:
[P-ts-1]: T 77394a9a5e254a619ec409feef132a4a P c630698bd61844ed8dd059f9b9b30d35 [R]: 1. PutCF: SubDocKey(DocKey(0xdaa5, [2777777], []), [SystemColumnId(0); HT{ days: 19471 time: 15:10:25.527681 }]) => null
[P-ts-1] I0424 15:10:25.544978 2112 tablet.cc:1502] T a380379d15f84ea39791f18cf5c20f3a P c630698bd61844ed8dd059f9b9b30d35: Wrote 7 key/value pairs to kIntents RocksDB:
[P-ts-1]: T a380379d15f84ea39791f18cf5c20f3a P c630698bd61844ed8dd059f9b9b30d35 [I]: 1. SingleDeleteCF: TXN META eed9b150-0180-498c-bee9-eb2072b3f12d
[P-ts-1]: T a380379d15f84ea39791f18cf5c20f3a P c630698bd61844ed8dd059f9b9b30d35 [I]: 2. SingleDeleteCF: TXN REV eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.399436 }
[P-ts-1]: T a380379d15f84ea39791f18cf5c20f3a P c630698bd61844ed8dd059f9b9b30d35 [I]: 3. SingleDeleteCF: SubDocKey(DocKey(0x2ead, [3777777], []), [SystemColumnId(0)]) [kStrongRead, kStrongWrite] HT{ days: 19471 time: 15:10:25.399436 }
[P-ts-1]: T a380379d15f84ea39791f18cf5c20f3a P c630698bd61844ed8dd059f9b9b30d35 [I]: 4. SingleDeleteCF: TXN REV eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.399436 w: 1 }
[P-ts-1]: T a380379d15f84ea39791f18cf5c20f3a P c630698bd61844ed8dd059f9b9b30d35 [I]: 5. SingleDeleteCF: SubDocKey(DocKey(0x2ead, [3777777], []), []) [kWeakRead, kWeakWrite] HT{ days: 19471 time: 15:10:25.399436 w: 1 }
[P-ts-1]: T a380379d15f84ea39791f18cf5c20f3a P c630698bd61844ed8dd059f9b9b30d35 [I]: 6. SingleDeleteCF: TXN REV eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.399436 w: 2 }
[P-ts-1]: T a380379d15f84ea39791f18cf5c20f3a P c630698bd61844ed8dd059f9b9b30d35 [I]: 7. SingleDeleteCF: SubDocKey(DocKey([], []), []) [kWeakRead, kWeakWrite] HT{ days: 19471 time: 15:10:25.399436 w: 2 }
[P-ts-1] I0424 15:10:25.545965 2877 tablet.cc:1502] T 77394a9a5e254a619ec409feef132a4a P c630698bd61844ed8dd059f9b9b30d35: Wrote 7 key/value pairs to kIntents RocksDB:
[P-ts-1]: T 77394a9a5e254a619ec409feef132a4a P c630698bd61844ed8dd059f9b9b30d35 [I]: 3. SingleDeleteCF: SubDocKey(DocKey(0xdaa5, [2777777], []), [SystemColumnId(0)]) [kStrongRead, kStrongWrite] HT{ days: 19471 time: 15:10:25.393740 }
[P-ts-1] I0424 15:10:25.548563 2390 tablet.cc:1502] T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35: Wrote 2 key/value pairs to kRegular RocksDB:
[P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [R]: 1. PutCF: SubDocKey(DocKey(0x56f1, [1777777], []), [SystemColumnId(0); HT{ days: 19471 time: 15:10:25.527681 }]) => null
[P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [R]: 2. PutCF: SubDocKey(DocKey(0x58db, [4777777], []), [SystemColumnId(0); HT{ days: 19471 time: 15:10:25.527681 w: 1 }]) => null
[P-ts-1] I0424 15:10:25.549944 2113 tablet.cc:1502] T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35: Wrote 13 key/value pairs to kIntents RocksDB:
[P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [I]: 3. SingleDeleteCF: SubDocKey(DocKey(0x56f1, [1777777], []), [SystemColumnId(0)]) [kStrongRead, kStrongWrite] HT{ days: 19471 time: 15:10:25.388093 }
[P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [I]: 9. SingleDeleteCF: SubDocKey(DocKey(0x58db, [4777777], []), [SystemColumnId(0)]) [kStrongRead, kStrongWrite] HT{ days: 19471 time: 15:10:25.411547 }
```
What about on a consumer tablet? We see:
```
# writing compressed external intents:
[C-ts-1] I0424 15:10:25.426849 2747 tablet.cc:1502] T 4221f0d394b94c5bbf9bcd4f29d20b35 P 5006739d83864a0789b86f3df1a6f800: Wrote 1 key/value pairs to kIntents RocksDB:
[C-ts-1]: T 4221f0d394b94c5bbf9bcd4f29d20b35 P 5006739d83864a0789b86f3df1a6f800 [I]: 1. PutCF: TXN EXT eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.393740 } => IT 77394a9a5e254a619ec409feef132a4a SubTransaction(2) [SubDocKey(DocKey(0xdaa5, [2777777], []), [SystemColumnId(0)]) -> null]
[C-ts-1] I0424 15:10:25.448917 2755 tablet.cc:1502] T 8db1600d22ee4855a1a024dc6ad59a39 P 5006739d83864a0789b86f3df1a6f800: Wrote 2 key/value pairs to kIntents RocksDB:
[C-ts-1]: T 8db1600d22ee4855a1a024dc6ad59a39 P 5006739d83864a0789b86f3df1a6f800 [I]: 1. PutCF: TXN EXT eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.388093 } => IT 04dc80f0149a43aa80204871d74d693a [SubDocKey(DocKey(0x56f1, [1777777], []), [SystemColumnId(0)]) -> null]
[C-ts-1]: T 8db1600d22ee4855a1a024dc6ad59a39 P 5006739d83864a0789b86f3df1a6f800 [I]: 2. PutCF: TXN EXT eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.388093 w: 1 } => IT 04dc80f0149a43aa80204871d74d693a SubTransaction(4) [SubDocKey(DocKey(0x58db, [4777777], []), [SystemColumnId(0)]) -> null]
[C-ts-1] I0424 15:10:25.471230 2970 tablet.cc:1502] T 02d87d4d5a754daeaa5ad8849c3e15c6 P 5006739d83864a0789b86f3df1a6f800: Wrote 1 key/value pairs to kIntents RocksDB:
[C-ts-1]: T 02d87d4d5a754daeaa5ad8849c3e15c6 P 5006739d83864a0789b86f3df1a6f800 [I]: 1. PutCF: TXN EXT eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.399436 } => IT a380379d15f84ea39791f18cf5c20f3a SubTransaction(3) [SubDocKey(DocKey(0x2ead, [3777777], []), [SystemColumnId(0)]) -> null]
# write in just the the un-aborted three values
[C-ts-1] I0424 15:10:25.562391 2755 tablet.cc:1502] T 8db1600d22ee4855a1a024dc6ad59a39 P 5006739d83864a0789b86f3df1a6f800: Wrote 2 key/value pairs to kRegular RocksDB:
[C-ts-1]: T 8db1600d22ee4855a1a024dc6ad59a39 P 5006739d83864a0789b86f3df1a6f800 [R]: 1. PutCF: SubDocKey(DocKey(0x56f1, [1777777], []), [SystemColumnId(0); HT{ days: 19471 time: 15:10:25.527681 }]) => null
[C-ts-1]: T 8db1600d22ee4855a1a024dc6ad59a39 P 5006739d83864a0789b86f3df1a6f800 [R]: 2. PutCF: SubDocKey(DocKey(0x58db, [4777777], []), [SystemColumnId(0); HT{ days: 19471 time: 15:10:25.527681 w: 1 }]) => null
[C-ts-1] I0424 15:10:25.646379 2755 tablet.cc:1502] T 4221f0d394b94c5bbf9bcd4f29d20b35 P 5006739d83864a0789b86f3df1a6f800: Wrote 1 key/value pairs to kRegular RocksDB:
[C-ts-1]: T 4221f0d394b94c5bbf9bcd4f29d20b35 P 5006739d83864a0789b86f3df1a6f800 [R]: 1. PutCF: SubDocKey(DocKey(0xdaa5, [2777777], []), [SystemColumnId(0); HT{ days: 19471 time: 15:10:25.527681 }]) => null
# deleting the compressed external intents:
[C-ts-1] I0424 15:10:25.562269 2755 tablet.cc:1502] T 8db1600d22ee4855a1a024dc6ad59a39 P 5006739d83864a0789b86f3df1a6f800: Wrote 2 key/value pairs to kIntents RocksDB:
[C-ts-1]: T 8db1600d22ee4855a1a024dc6ad59a39 P 5006739d83864a0789b86f3df1a6f800 [I]: 1. SingleDeleteCF: TXN EXT eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.388093 }
[C-ts-1]: T 8db1600d22ee4855a1a024dc6ad59a39 P 5006739d83864a0789b86f3df1a6f800 [I]: 2. SingleDeleteCF: TXN EXT eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.388093 w: 1 }
[C-ts-1] I0424 15:10:25.583690 2971 tablet.cc:1502] T 02d87d4d5a754daeaa5ad8849c3e15c6 P 5006739d83864a0789b86f3df1a6f800: Wrote 1 key/value pairs to kIntents RocksDB:
[C-ts-1]: T 02d87d4d5a754daeaa5ad8849c3e15c6 P 5006739d83864a0789b86f3df1a6f800 [I]: 1. SingleDeleteCF: TXN EXT eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.399436 }
[C-ts-1] I0424 15:10:25.646289 2755 tablet.cc:1502] T 4221f0d394b94c5bbf9bcd4f29d20b35 P 5006739d83864a0789b86f3df1a6f800: Wrote 1 key/value pairs to kIntents RocksDB:
[C-ts-1]: T 4221f0d394b94c5bbf9bcd4f29d20b35 P 5006739d83864a0789b86f3df1a6f800 [I]: 1. SingleDeleteCF: TXN EXT eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.393740 }
```
As you can see, the correct three of the four values got written on both sides.
Reviewers: hsunder, slingam, rahuldesirazu, xCluster, rsami
Reviewed By: hsunder, slingam, rsami
Subscribers: slingam, ybase, bogdan
Differential Revision: https://phorge.dev.yugabyte.com/D24763
mdbridge
added a commit
that referenced
this issue
May 18, 2023
…ectly handles SAVEPOINTs Summary: Original commit: c1ab3e7 / D24763 Changes made while cherry picking the original commit: * dropped changes to src/yb/integration-tests/xcluster_ysql-test.cc * this was just re-enabling the tests, which don't exist in 2.18 * rename dockv->docdb because that refactoring was not backported NOTE: the tests that accompanied the original version of this diff stack are not being back ported as part of this diff. Background: A savepoint is a special mark inside a transaction that allows all commands that are executed after it was established to be rolled back, restoring the transaction state to what it was at the time of the savepoint. In Yugabyte, Savepoints are implemented using sub-transactions, which are either committed or rolled back depending on whether user rolls back or releases the savepoint. Each intent belonging to a transaction is labeled with a subtransaction ID. As the transaction proceeds, the transaction manager keeps track of which subtransactions have been aborted so far. This set is incorporated in the COMMITTED Raft record when the transaction is committed. This information is then used by the apply transaction logic to figure out which intents should actually be applied. Before this diff, the subtransaction information, both for intents and for what was aborted, was not replicated across to the consumer side or used to do transaction applications. What this diff does at a high level: * in the CDC producer: * propagates the set of aborted subtransactions as a part of the APPLY CDC record * propagates with each intent what subtransaction it belongs to * on the receiving xCluster side: * place the what subtransactions were aborted in the apply external transactions info of the WriteRequestPB * likewise, encode the subtransaction ID of each intent * on the consumer tablet: * using the received information, apply only the external intents belonging to subtransactions that have not been aborted * add a new optional field of the compressed external intents format to store the subtransaction ID: ``` // External intents are stored in the following format: // key: kExternalTransactionId, txn_id // value: kUuid involved_tablet [kSubTransactionId subtransaction_ID] // kExternalIntents size(intent1_key), intent1_key, size(intent1_value), intent1_value, // size(intent2_key)... 0 // where size is encoded as varint. * we omit the subtransaction ID here and elsewhere when it is 1, the only subtransaction ID used in normal transactions that do not need multiple subtransactions ``` Notes on the new compressed intent format: * it supports rolling forward existing databases -- we still accept and use the original format * it will *not* directly support version downgrades as the old code will not understand the new format * intents are transient; likely it would be possible to create a run book to support rolling back * we use no extra space unless a transaction actually has multiple sub transactions * note that transactions using the default Postgres isolation level (READ COMMITTED RSN) will be using subtransactions if statements need to be retried * currently a fixed four bytes are being used for the subtransaction ID field plus a one byte prefix; probably a variable length encoding could save three bytes here if we thought it was needed What this diff does not do: * make SAVEPOINT's work with the pre-Hari scheme xCluster transactional replication Test Plan: (The described tests here have not been back ported) This diff enables the existing two failing tests for SAVEPOINT with transactional xCluster replication; they are now passing: ``` yb_build.sh --cxx-test xcluster_ysql-test --gtest_filter '*TransactionWithSavepoints*' ``` For better debugging messages right now, turn off packing: ``` yb_build.sh --cxx-test xcluster_ysql-test --gtest_filter '*TransactionWithSavepoints*' --test-args '--ysql_enable_packed_row=false' ``` I think you get the clearest sense of what's going on by looking at the intents of one producer TServer and one consumer TServer. For the test without optimization (TransactionWithSavepointsNoOpt), the SQL we are running is: ``` // Create two SAVEPOINTs but abort only one of them; all the writes except the aborted one should // be replicated and visible on the consumer side. ASSERT_OK(conn.Execute("BEGIN")); ASSERT_OK(conn.ExecuteFormat("insert into $0 values(1777777)", table_name)); ASSERT_OK(conn.Execute("SAVEPOINT a")); ASSERT_OK(conn.ExecuteFormat("insert into $0 values(2777777)", table_name)); ASSERT_OK(conn.Execute("RELEASE SAVEPOINT a")); ASSERT_OK(conn.Execute("SAVEPOINT b")); ASSERT_OK(conn.ExecuteFormat("insert into $0 values(3777777)", table_name)); ASSERT_OK(conn.Execute("ROLLBACK TO SAVEPOINT b")); ASSERT_OK(conn.ExecuteFormat("insert into $0 values(4777777)", table_name)); ASSERT_OK(conn.Execute("COMMIT")); ``` That results in the following (omitting uninteresting writes) on the producer tablet: ``` # intents in the uncompressed format for [1-4]777777, with corresponding subtransaction IDs # remember that we omit sub transaction ID 1. [P-ts-1] I0424 15:10:25.392364 2390 tablet.cc:1502] T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35: Wrote 7 key/value pairs to kIntents RocksDB: [P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [I]: 1. PutCF: TXN META eed9b150-0180-498c-bee9-eb2072b3f12d => { transaction_id: eed9b150-0180-498c-bee9-eb2072b3f12d isolation: SNAPSHOT_ISOLATION status_tablet: a13645c266084622bfe6e29190338041 priority: 15920732136751982613 start_time: { days: 19471 time: 15:10:25.380609 } locality: GLOBAL old_status_tablet: external_transaction: 0} [P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [I]: 2. PutCF: SubDocKey(DocKey(0x56f1, [1777777], []), [SystemColumnId(0)]) [kStrongRead, kStrongWrite] HT{ days: 19471 time: 15:10:25.388093 } => TransactionId(eed9b150-0180-498c-bee9-eb2072b3f12d) WriteId(0) null [P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [I]: 3. PutCF: TXN REV eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.388093 } => SubDocKey(DocKey(0x56f1, [1777777], []), [SystemColumnId(0)]) [kStrongRead, kStrongWrite] HT{ days: 19471 time: 15:10:25.388093 } [P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [I]: 4. PutCF: SubDocKey(DocKey(0x56f1, [1777777], []), []) [kWeakRead, kWeakWrite] HT{ days: 19471 time: 15:10:25.388093 w: 1 } => TransactionId(eed9b150-0180-498c-bee9-eb2072b3f12d) none [P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [I]: 5. PutCF: TXN REV eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.388093 w: 1 } => SubDocKey(DocKey(0x56f1, [1777777], []), []) [kWeakRead, kWeakWrite] HT{ days: 19471 time: 15:10:25.388093 w: 1 } [P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [I]: 6. PutCF: SubDocKey(DocKey([], []), []) [kWeakRead, kWeakWrite] HT{ days: 19471 time: 15:10:25.388093 w: 2 } => TransactionId(eed9b150-0180-498c-bee9-eb2072b3f12d) none [P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [I]: 7. PutCF: TXN REV eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.388093 w: 2 } => SubDocKey(DocKey([], []), []) [kWeakRead, kWeakWrite] HT{ days: 19471 time: 15:10:25.388093 w: 2 } [P-ts-1] I0424 15:10:25.396523 2845 tablet.cc:1502] T 77394a9a5e254a619ec409feef132a4a P c630698bd61844ed8dd059f9b9b30d35: Wrote 7 key/value pairs to kIntents RocksDB: [P-ts-1]: T 77394a9a5e254a619ec409feef132a4a P c630698bd61844ed8dd059f9b9b30d35 [I]: 2. PutCF: SubDocKey(DocKey(0xdaa5, [2777777], []), [SystemColumnId(0)]) [kStrongRead, kStrongWrite] HT{ days: 19471 time: 15:10:25.393740 } => TransactionId(eed9b150-0180-498c-bee9-eb2072b3f12d) SubTransactionId(2) WriteId(0) null [P-ts-1] I0424 15:10:25.403523 2390 tablet.cc:1502] T a380379d15f84ea39791f18cf5c20f3a P c630698bd61844ed8dd059f9b9b30d35: Wrote 7 key/value pairs to kIntents RocksDB: [P-ts-1]: T a380379d15f84ea39791f18cf5c20f3a P c630698bd61844ed8dd059f9b9b30d35 [I]: 2. PutCF: SubDocKey(DocKey(0x2ead, [3777777], []), [SystemColumnId(0)]) [kStrongRead, kStrongWrite] HT{ days: 19471 time: 15:10:25.399436 } => TransactionId(eed9b150-0180-498c-bee9-eb2072b3f12d) SubTransactionId(3) WriteId(0) null [P-ts-1] I0424 15:10:25.418630 2349 tablet.cc:1502] T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35: Wrote 6 key/value pairs to kIntents RocksDB: [P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [I]: 1. PutCF: SubDocKey(DocKey(0x58db, [4777777], []), [SystemColumnId(0)]) [kStrongRead, kStrongWrite] HT{ days: 19471 time: 15:10:25.411547 } => TransactionId(eed9b150-0180-498c-bee9-eb2072b3f12d) SubTransactionId(4) WriteId(1) null # start application: we write all of these except 3777777 and delete all the intents [P-ts-1] I0424 15:10:25.544713 2845 tablet.cc:1502] T 77394a9a5e254a619ec409feef132a4a P c630698bd61844ed8dd059f9b9b30d35: Wrote 1 key/value pairs to kRegular RocksDB: [P-ts-1]: T 77394a9a5e254a619ec409feef132a4a P c630698bd61844ed8dd059f9b9b30d35 [R]: 1. PutCF: SubDocKey(DocKey(0xdaa5, [2777777], []), [SystemColumnId(0); HT{ days: 19471 time: 15:10:25.527681 }]) => null [P-ts-1] I0424 15:10:25.544978 2112 tablet.cc:1502] T a380379d15f84ea39791f18cf5c20f3a P c630698bd61844ed8dd059f9b9b30d35: Wrote 7 key/value pairs to kIntents RocksDB: [P-ts-1]: T a380379d15f84ea39791f18cf5c20f3a P c630698bd61844ed8dd059f9b9b30d35 [I]: 1. SingleDeleteCF: TXN META eed9b150-0180-498c-bee9-eb2072b3f12d [P-ts-1]: T a380379d15f84ea39791f18cf5c20f3a P c630698bd61844ed8dd059f9b9b30d35 [I]: 2. SingleDeleteCF: TXN REV eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.399436 } [P-ts-1]: T a380379d15f84ea39791f18cf5c20f3a P c630698bd61844ed8dd059f9b9b30d35 [I]: 3. SingleDeleteCF: SubDocKey(DocKey(0x2ead, [3777777], []), [SystemColumnId(0)]) [kStrongRead, kStrongWrite] HT{ days: 19471 time: 15:10:25.399436 } [P-ts-1]: T a380379d15f84ea39791f18cf5c20f3a P c630698bd61844ed8dd059f9b9b30d35 [I]: 4. SingleDeleteCF: TXN REV eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.399436 w: 1 } [P-ts-1]: T a380379d15f84ea39791f18cf5c20f3a P c630698bd61844ed8dd059f9b9b30d35 [I]: 5. SingleDeleteCF: SubDocKey(DocKey(0x2ead, [3777777], []), []) [kWeakRead, kWeakWrite] HT{ days: 19471 time: 15:10:25.399436 w: 1 } [P-ts-1]: T a380379d15f84ea39791f18cf5c20f3a P c630698bd61844ed8dd059f9b9b30d35 [I]: 6. SingleDeleteCF: TXN REV eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.399436 w: 2 } [P-ts-1]: T a380379d15f84ea39791f18cf5c20f3a P c630698bd61844ed8dd059f9b9b30d35 [I]: 7. SingleDeleteCF: SubDocKey(DocKey([], []), []) [kWeakRead, kWeakWrite] HT{ days: 19471 time: 15:10:25.399436 w: 2 } [P-ts-1] I0424 15:10:25.545965 2877 tablet.cc:1502] T 77394a9a5e254a619ec409feef132a4a P c630698bd61844ed8dd059f9b9b30d35: Wrote 7 key/value pairs to kIntents RocksDB: [P-ts-1]: T 77394a9a5e254a619ec409feef132a4a P c630698bd61844ed8dd059f9b9b30d35 [I]: 3. SingleDeleteCF: SubDocKey(DocKey(0xdaa5, [2777777], []), [SystemColumnId(0)]) [kStrongRead, kStrongWrite] HT{ days: 19471 time: 15:10:25.393740 } [P-ts-1] I0424 15:10:25.548563 2390 tablet.cc:1502] T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35: Wrote 2 key/value pairs to kRegular RocksDB: [P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [R]: 1. PutCF: SubDocKey(DocKey(0x56f1, [1777777], []), [SystemColumnId(0); HT{ days: 19471 time: 15:10:25.527681 }]) => null [P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [R]: 2. PutCF: SubDocKey(DocKey(0x58db, [4777777], []), [SystemColumnId(0); HT{ days: 19471 time: 15:10:25.527681 w: 1 }]) => null [P-ts-1] I0424 15:10:25.549944 2113 tablet.cc:1502] T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35: Wrote 13 key/value pairs to kIntents RocksDB: [P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [I]: 3. SingleDeleteCF: SubDocKey(DocKey(0x56f1, [1777777], []), [SystemColumnId(0)]) [kStrongRead, kStrongWrite] HT{ days: 19471 time: 15:10:25.388093 } [P-ts-1]: T 04dc80f0149a43aa80204871d74d693a P c630698bd61844ed8dd059f9b9b30d35 [I]: 9. SingleDeleteCF: SubDocKey(DocKey(0x58db, [4777777], []), [SystemColumnId(0)]) [kStrongRead, kStrongWrite] HT{ days: 19471 time: 15:10:25.411547 } ``` What about on a consumer tablet? We see: ``` # writing compressed external intents: [C-ts-1] I0424 15:10:25.426849 2747 tablet.cc:1502] T 4221f0d394b94c5bbf9bcd4f29d20b35 P 5006739d83864a0789b86f3df1a6f800: Wrote 1 key/value pairs to kIntents RocksDB: [C-ts-1]: T 4221f0d394b94c5bbf9bcd4f29d20b35 P 5006739d83864a0789b86f3df1a6f800 [I]: 1. PutCF: TXN EXT eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.393740 } => IT 77394a9a5e254a619ec409feef132a4a SubTransaction(2) [SubDocKey(DocKey(0xdaa5, [2777777], []), [SystemColumnId(0)]) -> null] [C-ts-1] I0424 15:10:25.448917 2755 tablet.cc:1502] T 8db1600d22ee4855a1a024dc6ad59a39 P 5006739d83864a0789b86f3df1a6f800: Wrote 2 key/value pairs to kIntents RocksDB: [C-ts-1]: T 8db1600d22ee4855a1a024dc6ad59a39 P 5006739d83864a0789b86f3df1a6f800 [I]: 1. PutCF: TXN EXT eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.388093 } => IT 04dc80f0149a43aa80204871d74d693a [SubDocKey(DocKey(0x56f1, [1777777], []), [SystemColumnId(0)]) -> null] [C-ts-1]: T 8db1600d22ee4855a1a024dc6ad59a39 P 5006739d83864a0789b86f3df1a6f800 [I]: 2. PutCF: TXN EXT eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.388093 w: 1 } => IT 04dc80f0149a43aa80204871d74d693a SubTransaction(4) [SubDocKey(DocKey(0x58db, [4777777], []), [SystemColumnId(0)]) -> null] [C-ts-1] I0424 15:10:25.471230 2970 tablet.cc:1502] T 02d87d4d5a754daeaa5ad8849c3e15c6 P 5006739d83864a0789b86f3df1a6f800: Wrote 1 key/value pairs to kIntents RocksDB: [C-ts-1]: T 02d87d4d5a754daeaa5ad8849c3e15c6 P 5006739d83864a0789b86f3df1a6f800 [I]: 1. PutCF: TXN EXT eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.399436 } => IT a380379d15f84ea39791f18cf5c20f3a SubTransaction(3) [SubDocKey(DocKey(0x2ead, [3777777], []), [SystemColumnId(0)]) -> null] # write in just the the un-aborted three values [C-ts-1] I0424 15:10:25.562391 2755 tablet.cc:1502] T 8db1600d22ee4855a1a024dc6ad59a39 P 5006739d83864a0789b86f3df1a6f800: Wrote 2 key/value pairs to kRegular RocksDB: [C-ts-1]: T 8db1600d22ee4855a1a024dc6ad59a39 P 5006739d83864a0789b86f3df1a6f800 [R]: 1. PutCF: SubDocKey(DocKey(0x56f1, [1777777], []), [SystemColumnId(0); HT{ days: 19471 time: 15:10:25.527681 }]) => null [C-ts-1]: T 8db1600d22ee4855a1a024dc6ad59a39 P 5006739d83864a0789b86f3df1a6f800 [R]: 2. PutCF: SubDocKey(DocKey(0x58db, [4777777], []), [SystemColumnId(0); HT{ days: 19471 time: 15:10:25.527681 w: 1 }]) => null [C-ts-1] I0424 15:10:25.646379 2755 tablet.cc:1502] T 4221f0d394b94c5bbf9bcd4f29d20b35 P 5006739d83864a0789b86f3df1a6f800: Wrote 1 key/value pairs to kRegular RocksDB: [C-ts-1]: T 4221f0d394b94c5bbf9bcd4f29d20b35 P 5006739d83864a0789b86f3df1a6f800 [R]: 1. PutCF: SubDocKey(DocKey(0xdaa5, [2777777], []), [SystemColumnId(0); HT{ days: 19471 time: 15:10:25.527681 }]) => null # deleting the compressed external intents: [C-ts-1] I0424 15:10:25.562269 2755 tablet.cc:1502] T 8db1600d22ee4855a1a024dc6ad59a39 P 5006739d83864a0789b86f3df1a6f800: Wrote 2 key/value pairs to kIntents RocksDB: [C-ts-1]: T 8db1600d22ee4855a1a024dc6ad59a39 P 5006739d83864a0789b86f3df1a6f800 [I]: 1. SingleDeleteCF: TXN EXT eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.388093 } [C-ts-1]: T 8db1600d22ee4855a1a024dc6ad59a39 P 5006739d83864a0789b86f3df1a6f800 [I]: 2. SingleDeleteCF: TXN EXT eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.388093 w: 1 } [C-ts-1] I0424 15:10:25.583690 2971 tablet.cc:1502] T 02d87d4d5a754daeaa5ad8849c3e15c6 P 5006739d83864a0789b86f3df1a6f800: Wrote 1 key/value pairs to kIntents RocksDB: [C-ts-1]: T 02d87d4d5a754daeaa5ad8849c3e15c6 P 5006739d83864a0789b86f3df1a6f800 [I]: 1. SingleDeleteCF: TXN EXT eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.399436 } [C-ts-1] I0424 15:10:25.646289 2755 tablet.cc:1502] T 4221f0d394b94c5bbf9bcd4f29d20b35 P 5006739d83864a0789b86f3df1a6f800: Wrote 1 key/value pairs to kIntents RocksDB: [C-ts-1]: T 4221f0d394b94c5bbf9bcd4f29d20b35 P 5006739d83864a0789b86f3df1a6f800 [I]: 1. SingleDeleteCF: TXN EXT eed9b150-0180-498c-bee9-eb2072b3f12d HT{ days: 19471 time: 15:10:25.393740 } ``` As you can see, the correct three of the four values got written on both sides. Reviewers: hsunder, slingam, rahuldesirazu, xCluster Reviewed By: hsunder Subscribers: bogdan, ybase, slingam Differential Revision: https://phorge.dev.yugabyte.com/D25430
|
backported to 2.18.1 |
mdbridge
added a commit
that referenced
this issue
Jul 24, 2023
… XCluster
Summary:
See title.
Also changed this flag stability level from evolving to stable.
Jira: DB-3745
Test Plan:
```
$ yb-master --help|grep -A 1 savepoint
-enable_pg_savepoints (Set to false to disable savepoints in YugaByte
PostgreSQL API.) type: bool default: true
```
Reviewers: xCluster, hsunder
Reviewed By: hsunder
Subscribers: yql, ybase
Differential Revision: https://phorge.dev.yugabyte.com/D27199
mdbridge
added a commit
that referenced
this issue
Jul 25, 2023
…oints work with XCluster Summary: Original commit: b924861 / D27199 Clean merge to origin/2.18. original summary See title. Also changed this flag stability level from evolving to stable. Jira: DB-3745 Test Plan: ``` $ yb-master --help|grep -A 1 savepoint -enable_pg_savepoints (Set to false to disable savepoints in YugaByte PostgreSQL API.) type: bool default: true ``` Reviewers: xCluster, hsunder Reviewed By: hsunder Subscribers: ybase, yql Differential Revision: https://phorge.dev.yugabyte.com/D27213
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Jira Link: DB-3745
Description
A savepoint is a special mark inside a transaction that allows all commands that are executed after it was established to be rolled back, restoring the transaction state to what it was at the time of the savepoint.
In Yugabyte, Savepoints are implemented using sub-transactions which are either committed or rolled back depending on whether user rollsback or releases savepoint.
As part of transactional xcluster replication, the WAL records from the system.transactions are replicated from the producer to the consumer universe, however the list of aborted/committed sub-transactions is currently not propagated.
To support SAVEPOINTs, changes will be needed to propagate the list of aborted sub-transactions from the Producer->Consumer and propagate that forward to the corresponding transaction coordinator and participants as part of UpdateTransaction.
The text was updated successfully, but these errors were encountered: