txn: avoid useless calculation and wake-up msgs #4813

youjiali1995 · 2019-05-30T16:35:03Z

Signed-off-by: youjiali1995 zlwgx1023@gmail.com

What have you changed? (mandatory)

Add atomic variable WAIT_TABLE_IS_EMPTY. If it is true, the wait table is empty and there is no need to calculate keys' hashes and wake up waiters.

What are the type of the changes? (mandatory)

Improvement (change which is an improvement to an existing feature)

How has this PR been tested? (mandatory)

Please describe the tests that you ran to verify your changes. Have you finished unit tests, integration tests, or manual tests? What additional tests would give you greater confidence in this change?

Does this PR affect documentation (docs) or release note? (mandatory)

No

Does this PR affect tidb-ansible update? (mandatory)

No

Add a few positive/negative examples (optional)

Now there is no wake-up msg sent to waiter manager if only uses the optimistic transaction.


        pessimistic-txn: avoid useless wake-up msgs if wait talbe is empty

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>


        bugfix

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>


        Merge branch 'master' into avoid-useless-wake-up

siddontang · 2019-05-30T20:41:31Z

src/storage/lock_manager/waiter_manager.rs

@@ -91,6 +95,9 @@ impl WaitTable {
    }

    fn add_waiter(&mut self, ts: u64, waiter: Waiter) -> bool {
+        if self.wait_table.is_empty() {


better to abstract this to a function.


        fix conflict

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>


        address comment

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>


        Merge branch 'master' into avoid-useless-wake-up

youjiali1995 · 2019-05-31T07:19:14Z

@coocood @AndreMouche @zhangjinpeng1987 PTAL

src/storage/lock_manager/waiter_manager.rs


        fix data race

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>


        Merge branch 'avoid-useless-wake-up' of github.com:youjiali1995/tikv …

…into avoid-useless-wake-up


        fix test

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

coocood · 2019-05-31T09:38:23Z

LGTM


        Merge branch 'master' into avoid-useless-wake-up

youjiali1995 · 2019-06-04T05:26:32Z

@zhangjinpeng1987 @AndreMouche PTAL

src/storage/lock_manager/waiter_manager.rs

zhangjinpeng1987 · 2019-06-05T03:46:44Z

src/storage/txn/process.rs

-            } else {
-                None
-            };
+            let key_hashes = gen_key_hashes_if_needed(&waiter_mgr_scheduler, &keys);


Every commit will load atomic variable, can we avoid it?

src/storage/txn/process.rs

src/storage/txn/scheduler.rs


        address comment

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>


        Merge branch 'master' into avoid-useless-wake-up

zhangjinpeng1987 · 2019-06-05T05:38:54Z

src/storage/lock_manager/waiter_manager.rs

+    WAIT_TABLE_IS_EMPTY.store(is_empty, Ordering::Relaxed);
+}
+
+pub fn load_wait_table_is_empty() -> bool {


s/load_wait_table_is_empty/wait_table_is_empty


        address comment

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>


        Merge branch 'avoid-useless-wake-up' of github.com:youjiali1995/tikv …

…into avoid-useless-wake-up

zhangjinpeng1987

LGTM


        Merge branch 'master' of github.com:tikv/tikv into avoid-useless-wake-up

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

AndreMouche

Rest LGTM

AndreMouche · 2019-06-06T12:49:04Z

src/storage/txn/process.rs

-            notify_waiter_mgr(&waiter_mgr_scheduler, lock_ts, key_hashes, commit_ts);
-            notify_deadlock_detector(&detector_scheduler, is_pessimistic_txn, lock_ts);
+            notify_waiter_mgr_if_needed(&waiter_mgr_scheduler, lock_ts, key_hashes, commit_ts);
+            notify_deadlock_detector_if_needed(&detector_scheduler, is_pessimistic_txn, lock_ts);


Could we merge these two functions into one since they are always used together? Maybe define a new function which call these two functions?

The function needs many parameters, and I think it is a bit redundant.

AndreMouche · 2019-06-06T12:49:04Z

src/storage/txn/process.rs

+                            if wait_table_is_empty {
+                                None
+                            } else {
+                                Some(vec![])


I think we should insert current keys into the empty vec here.

Nice catch!


        bugfix

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>


        Merge branch 'master' into avoid-useless-wake-up


        Merge branch 'master' into avoid-useless-wake-up


        Merge branch 'master' into avoid-useless-wake-up

src/storage/txn/process.rs

AndreMouche

Rest LGTM

src/storage/txn/process.rs


        address comment

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

youjiali1995 · 2019-06-10T07:53:37Z

@AndreMouche @zhangjinpeng1987 PTAL

zhangjinpeng1987

LGTM


        txn: avoid useless calculation and wake-up msgs (tikv#4813)

* pessimistic-txn: avoid useless wake-up msgs if wait talbe is empty Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * bugfix Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * fix conflict Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * address comment Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * fix data race Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * fix test Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * address comment Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * address comment Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * bugfix Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * address comment Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>


        txn: avoid useless calculation and wake-up msgs (tikv#4813)

* pessimistic-txn: avoid useless wake-up msgs if wait talbe is empty Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * bugfix Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * fix conflict Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * address comment Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * fix data race Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * fix test Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * address comment Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * address comment Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * bugfix Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * address comment Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>


        txn: avoid useless calculation and wake-up msgs (tikv#4813)

* pessimistic-txn: avoid useless wake-up msgs if wait talbe is empty Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * bugfix Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * fix conflict Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * address comment Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * fix data race Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * fix test Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * address comment Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * address comment Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * bugfix Signed-off-by: youjiali1995 <zlwgx1023@gmail.com> * address comment Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

youjiali1995 added 3 commits May 30, 2019

pessimistic-txn: avoid useless wake-up msgs if wait talbe is empty

5d7e3c1

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

bugfix

Loading status checks…

dbdadc0

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

Merge branch 'master' into avoid-useless-wake-up

Verified

This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.

GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits

Loading status checks…

e423226

siddontang reviewed May 30, 2019

View changes

youjiali1995 added 3 commits May 31, 2019

fix conflict

Loading status checks…

32041fa

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

address comment

Loading status checks…

3e7efcc

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

Merge branch 'master' into avoid-useless-wake-up

Verified

This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.

GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits

Loading status checks…

969444b

coocood reviewed May 31, 2019

View changes

src/storage/lock_manager/waiter_manager.rs Outdated Show resolved Hide resolved

youjiali1995 added 3 commits May 31, 2019

fix data race

571a36d

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

Merge branch 'avoid-useless-wake-up' of github.com:youjiali1995/tikv …

Loading status checks…

51a2d45

…into avoid-useless-wake-up

fix test

Loading status checks…

061af6d

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

youjiali1995 requested review from zhangjinpeng1987 and AndreMouche May 31, 2019

Merge branch 'master' into avoid-useless-wake-up

Verified

This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.

GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits

Loading status checks…

f302fb3

zhangjinpeng1987 reviewed Jun 5, 2019

View changes

src/storage/lock_manager/waiter_manager.rs Outdated Show resolved Hide resolved

zhangjinpeng1987 reviewed Jun 5, 2019

View changes

src/storage/txn/process.rs

} else {

None

};

let key_hashes = gen_key_hashes_if_needed(&waiter_mgr_scheduler, &keys);

This comment has been minimized.

Sign in to view

Copy link

zhangjinpeng1987 Jun 5, 2019
Member

Every commit will load atomic variable, can we avoid it?

zhangjinpeng1987 reviewed Jun 5, 2019

View changes

src/storage/txn/process.rs Outdated Show resolved Hide resolved

zhangjinpeng1987 reviewed Jun 5, 2019

View changes

src/storage/txn/scheduler.rs Outdated Show resolved Hide resolved

youjiali1995 added 2 commits Jun 5, 2019

address comment

Loading status checks…

d464256

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

Merge branch 'master' into avoid-useless-wake-up

Verified

This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.

GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits

Loading status checks…

625d6d0

zhangjinpeng1987 reviewed Jun 5, 2019

View changes

youjiali1995 added 2 commits Jun 5, 2019

address comment

e07000a

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

Merge branch 'avoid-useless-wake-up' of github.com:youjiali1995/tikv …

Loading status checks…

9ec1f93

…into avoid-useless-wake-up

zhangjinpeng1987 reviewed Jun 5, 2019

View changes

youjiali1995 added the S: LGT1 label Jun 6, 2019

Merge branch 'master' of github.com:tikv/tikv into avoid-useless-wake-up

Loading status checks…

f96bd6c

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

youjiali1995 dismissed zhangjinpeng1987’s stale review via f96bd6c Jun 6, 2019

AndreMouche self-assigned this Jun 6, 2019

AndreMouche reviewed Jun 6, 2019

View changes

youjiali1995 added 4 commits Jun 6, 2019

bugfix

Loading status checks…

e86aac8

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

Merge branch 'master' into avoid-useless-wake-up

Verified

This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.

GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits

Loading status checks…

a31d684

Merge branch 'master' into avoid-useless-wake-up

Verified

This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.

GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits

Loading status checks…

c104892

Merge branch 'master' into avoid-useless-wake-up

Verified

This commit was created on GitHub.com and signed with a verified signature using GitHub’s key.

GPG key ID: 4AEE18F83AFDEB23 Learn about signing commits

Loading status checks…

48a4f83

AndreMouche reviewed Jun 10, 2019

View changes

src/storage/txn/process.rs Show resolved Hide resolved

AndreMouche reviewed Jun 10, 2019

View changes

src/storage/txn/process.rs Outdated Show resolved Hide resolved

address comment

Loading status checks…

acc89d1

Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>

youjiali1995 force-pushed the youjiali1995:avoid-useless-wake-up branch from 0d87c69 to acc89d1 Jun 10, 2019

AndreMouche approved these changes Jun 10, 2019

View changes

zhangjinpeng1987 approved these changes Jun 10, 2019

View changes

youjiali1995 merged commit de35885 into tikv:master Jun 10, 2019
2 checks passed

2 checks passed

DCO All commits are signed off!
Details

idc-jenkins-ci/test Jenkins job succeeded.
Details

youjiali1995 deleted the youjiali1995:avoid-useless-wake-up branch Jun 10, 2019

tikv/tikv

txn: avoid useless calculation and wake-up msgs #4813

txn: avoid useless calculation and wake-up msgs #4813

youjiali1995 commented May 30, 2019

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

youjiali1995 commented May 31, 2019

This comment has been minimized.

coocood commented May 31, 2019

This comment has been minimized.

youjiali1995 commented Jun 4, 2019

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

zhangjinpeng1987 left a comment

AndreMouche left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

AndreMouche left a comment

This comment has been minimized.

youjiali1995 commented Jun 10, 2019

zhangjinpeng1987 left a comment

tikv/tikv

Join GitHub today

txn: avoid useless calculation and wake-up msgs #4813

txn: avoid useless calculation and wake-up msgs #4813

Conversation

youjiali1995 commented May 30, 2019

What have you changed? (mandatory)

What are the type of the changes? (mandatory)

How has this PR been tested? (mandatory)

Does this PR affect documentation (docs) or release note? (mandatory)

Does this PR affect tidb-ansible update? (mandatory)

Add a few positive/negative examples (optional)

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

youjiali1995 commented May 31, 2019

This comment has been minimized.

coocood commented May 31, 2019

This comment has been minimized.

youjiali1995 commented Jun 4, 2019

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

zhangjinpeng1987 left a comment

AndreMouche left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

AndreMouche left a comment

This comment has been minimized.

youjiali1995 commented Jun 10, 2019

zhangjinpeng1987 left a comment