Skip to content

2.25.2.0-b55

@basavaraj29 basavaraj29 tagged this 26 Feb 04:28
Summary:
**Background**
Distributed transactions
In YB, we don't always create an explicit docdb distributed transaction unless required. For instance,
1. RC/RR txns in explicit transactional block with no reads with row locks or updates/writes
2. fast-path/single-shard writes etc.
Since the above don't write/request intents, a docdb txn wasn't required.

**Problem**
We want to introduce support for table locks which would mean that we need some identifier to tag the locks to. Also, there could be deadlocks spanning row-level locks & object/table locks & advisory locks. So we would need to tie the object locks to the same docdb transaction.

**Idea**
We could start a distributed docdb transaction early (even for read-only txns), and keep re-using the transaction until either of the following happen:
- a transaction with intents consumes it
- a transaction that uses savepoints consumes it (excluding read committed isolation txns for now)
- if the transaction ends up blocking a DDL (this would help prevent false deadlock issue since we would be reusing read only txns)

The next subsequent statement would start another docdb transaction.

That way, we would have an identifier to tag object/table locks to. For instance, consider the below example
```
ysqlsh>                       // create TXN1
begin;
select * from t ...;          // create TXN1. use <TXN1, subtxn_id> for object locks
commit;                       // was a read only txn, reuse TXN1 if it satisfies all conditions
update t set v=v+1 where k=1; // use <TXN1, subtxn_id> for object locks
                              // fast path txn, so wouldn't have intents. can reuse TXN1
begin;
select * from t ...;          // use <TXN1, subtxn_id> for object locks
update t set ...;             // we use <TXN1> to write intents, so the txn cannot be reused
commit;                       // TXN1 gets consumed
select * from t ...;          // create txn TXN2, and use <TXN2, subtxn_id> for object locks
```

The basic infrastructure piece for txn reuse was introduced in https://phorge.dev.yugabyte.com/D41282

This diff adds the following changes. When `TEST_enable_object_locking_for_table_locks` is enabled
- object lock calls are forwarded from `pggate` to `pg_client_service` layer. Additionally, txn finish calls are forwarded from `pggate` for read-only statements/txns too.
- shared locks are acquired on the local tserver, so essentially all DMLs would acquire object/table locks when the flag is set.
- on statement/transaction finish, the local object locks are released in-line.

The test flag is disabled by default, and hence the diff shouldn't cause any semantic change in behavior on both docdb and ysql ends.

**DocDB changes that can go as subsequent diff**
1. Forward exclusive object locks to the master to acquire them globally.
2. In YB, READ_COMMITTED isolation internally leverages savepoints. Enabling the above would lead to burn of 1 docdb transaction for every RC read-only txn, which is not desirable. Hence for RC alone, this change ignores creating a doc dist txn on savepoint => object locks wouldn't be rolled back on rollback to savepoint if the txn hasn't written any intents until then (those object locks gets released only on finish). Fix behavior for RC.
3. Investigate failures with ParallelQuery and extended query mode execution when the test flag is enabled.

**YSQL changes that might help test the diff better**
1. Need to expose something similar to `yb_get_current_transaction` that would help us add more tests for this change using `PGConn`. Modifying `yb_get_current_transaction` didn't seem like the correct approach as it could end up showing the same txn id across different YSQL read-only txns.

**Upgrade/Downgrade safety**
No upgrade/downgrade issues since the only proto changes are for local node communication between pggate and pg_client_service.
Jira: DB-15217

Test Plan: Jenkins

Reviewers: dmitry, pjain

Reviewed By: dmitry

Subscribers: yql, ybase

Differential Revision: https://phorge.dev.yugabyte.com/D39894
Assets 2
Loading