txn: Add txn state's view

[longfangsong]

What problem does this PR solve?

Problem Summary:

Currently, there're no way to view the state of a transaction with SQL, which sometimes makes diagnosing related problems hard.

/cc #24199

What is changed and how it works?

Proposal: TiDB Transaction related views

What's Changed:

Add a (CLUSTER_)TIDB_TRX to view the transaction state.

How it Works:

Just create a memTable, and query the data from all session's TxnState on query.

Use Example

use test; create table t(id int, a varchar(128)); 
insert into t(id, a) values (1, 'abcd'); 
BEGIN; 
select * from t for update; 
select * from information_schema.CLUSTER_TIDB_TRX;

Will show:

+-----------------+--------------------+---------------------+------------------------------------------------------------------+--------+--------------------+------+------+------------+------+------+
| INSTANCE        | ID                 | START_TIME          | DIGEST                                                           | STATE  | WAITING_START_TIME | LEN  | SIZE | SESSION_ID | USER | DB   |
+-----------------+--------------------+---------------------+------------------------------------------------------------------+--------+--------------------+------+------+------------+------+------+
| 127.0.0.1:10080 | 424768545227014155 | 2021-05-07 12:56:48 | a200e35b257452d27bbad7f5cb67435a0f5804848baafbc6674212c917816245 | Normal | NULL               |    1 |   19 |          7 | root | test |
+-----------------+--------------------+---------------------+------------------------------------------------------------------+--------+--------------------+------+------+------------+------+------+
1 row in set (0.00 sec)

Related changes

• PR to update pingcap/docs/pingcap/docs-cn (Will update after this pr is ready for merge.)

Check List

Tests

• Unit test

Side effects

• Performance regression
  • Consumes more CPU
  • Consumes more MEM

Release note

• Add a (CLUSTER_)TIDB_TRX to view the transaction state.

[longfangsong]

/sig transaction

[cfzjywxk]

/run-all-tests

[longfangsong]

/run-all-tests

[longfangsong]

/cc @MyonKeminta @cfzjywxk
Instead of make memory buffer's field atomic, I collect Len() and Size() and update corresponding fields in LazyTxn atomically on each write of memory buffer, and we have no races detected now.
（The failed intergration tests seems has nothing to do with this pr and will be solved by #24052）

[MyonKeminta]

Is the size the number of total bytes? I just realize that if a transaction's size exceeds 4GB, it's size may not fit in a int value. It's out of the topic of this PR though.

[MyonKeminta]

LGTM

[MyonKeminta]

Note that there's still an unresolved problem in this PR: the sql digest and the txn state/wait time is not fetched at the same time and without any lock. So it's possible that after getting the txn's state, the transaction starts executing the next statement so another sql digest is fetched. causing inconsistency in the result columns. This still needs to be optimized in the future.

[cfzjywxk]

/lgtm

[cfzjywxk]

/merge

[ti-chi-bot]

This pull request has been accepted and is ready to merge.

Commit hash: 1015efc

[cfzjywxk]

Note that there's still an unresolved problem in this PR: the sql digest and the txn state/wait time is not fetched at the same time and without any lock. So it's possible that after getting the txn's state, the transaction starts executing the next statement so another sql digest is fetched. causing inconsistency in the result columns. This still needs to be optimized in the future.

@MyonKeminta we could record a development task for it.

[MyonKeminta]

Note that there's still an unresolved problem in this PR: the sql digest and the txn state/wait time is not fetched at the same time and without any lock. So it's possible that after getting the txn's state, the transaction starts executing the next statement so another sql digest is fetched. causing inconsistency in the result columns. This still needs to be optimized in the future.

@MyonKeminta we could record a development task for it.

Ok. I've directly recorded it in the tracking issue

[ti-chi-bot]

/run-all-tests

This bot automatically retries jobs that failed on can merge PRs (send feedback to hi-rustin).

Silence the bot with the /merge cancel comment for consistent failures.

[Linda-wang00]

Hi Thank you for your excellent services!

[wuhuizuo]

/review default

[ti-chi-bot]

@wuhuizuo:
Review of PR #1234

General

This pull request introduces some changes to the txn package and util package. It adds functionality for tracking transaction information, such as the state of a transaction, the number of entries in memory, and other useful information. It also includes some refactoring of the code in the txn package to improve the locking mechanism and optimize some memory usage.

Specific

txn/lazytxn.go

• LockKeys - This method adds additional tracking of the state of the transaction and stores a timestamp when a lock is blocked. It also updates the txn info structure's EntriesCount and EntriesSize variables. This method appears to be well-implemented and has helpful changes.
• reset - The method here changes the receiver to LazyTxn rather than TxnState. This change makes sense given the other changes made in this pull request. The cleanup code looks like mostly renaming of variables and seems to be modified appropriately.
• KeysNeedToLock - The changes to this method look fine. The receiver has been changed to LazyTxn and the implementation has been updated slightly, but the functionality appears to have remained the same.
• Info - A new method, this adds helpful functionality to the txn package. It populates and returns a txn info structure, which can be used to display transaction information in an informative way.
• UpdateEntriesCountAndSize - This method also adds helpful functionality. It updates the number and size of entries in the MemDB.

txn/txninfo/txn_info.go

• This file adds the txninfo package, which includes some new structs and functions that are used to populate transaction information in a way that is useful for users.

util/processinfo.go

• SessionManager - the interface has been updated to add a ShowTxnList method that returns an array of txninfo structs.

Conclusion

This pull request adds useful functionality for tracking transaction information, and updates the implementaion to improve the locking mechanism and optimize memory usage. The changes seem well-designed and implemented appropriately. I recommend approving this pull request.

In response to this:

/review default

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.