Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

*: support pessimistic transaction (experimental feature) #10297

Merged
merged 13 commits into from May 11, 2019

Conversation

Projects
None yet
5 participants
@coocood
Copy link
Member

commented Apr 28, 2019

What problem does this PR solve?

Avoid transaction failure caused by write conflict, especially when the transaction cannot be retried automatically.

What is changed and how it works?

  • Use BEGIN LOCK syntax to start a new pessimistic transaction.
  • In the pessimistic transaction, write pessimistic lock on INSERT/UPDATE/DELETE/SELECT FOR UPDATE.
  • Pessimistic locks don't have value, don't block read.
  • Pessimistic locks blocks writes to make sure there is no write conflict during Prewrite phase.

Check List

Tests

  • Unit test
  • Integration test

Code changes

  • Has exported function/method change
  • Has exported variable/fields change
  • Has interface methods change
  • Has persistent data change

Side effects

  • Possible performance regression
  • Increased code complexity

Related changes

  • Need to update the documentation
  • Need to be included in the release note

coocood added some commits Apr 28, 2019

@codecov

This comment has been minimized.

Copy link

commented Apr 28, 2019

Codecov Report

❗️ No coverage uploaded for pull request base (master@5617809). Click here to learn what that means.
The diff coverage is 21.8142%.

@@            Coverage Diff             @@
##             master    #10297   +/-   ##
==========================================
  Coverage          ?   77.012%           
==========================================
  Files             ?       412           
  Lines             ?     86232           
  Branches          ?         0           
==========================================
  Hits              ?     66409           
  Misses            ?     14754           
  Partials          ?      5069
@coocood

This comment has been minimized.

Copy link
Member Author

commented Apr 28, 2019

/run-all-tests

@coocood coocood force-pushed the coocood:pessimistic-txn branch from b0ef687 to 41d79c7 Apr 29, 2019

@jackysp jackysp changed the title support pessimistic transaction (experimental feature) *: support pessimistic transaction (experimental feature) Apr 29, 2019

Show resolved Hide resolved config/config.go Outdated
@zhouqiang-cl

This comment has been minimized.

Copy link
Member

commented Apr 29, 2019

/run-all-tests tidb-test=pr/795

Show resolved Hide resolved config/config.go
Show resolved Hide resolved config/config.toml.example Outdated
Show resolved Hide resolved executor/adapter.go Outdated
Show resolved Hide resolved executor/adapter.go Outdated
Show resolved Hide resolved executor/adapter.go

func (s *testSessionSuite) TestPessimisticTxn(c *C) {
if !config.GetGlobalConfig().PessimisticTxn.Enable {
c.Skip("pessimistic transaction is not enabled")

This comment has been minimized.

Copy link
@tiancaiamao

tiancaiamao Apr 29, 2019

Contributor

The default conf is false, this test will never run?!

This comment has been minimized.

Copy link
@coocood

coocood Apr 30, 2019

Author Member

Yes.

Show resolved Hide resolved session/session_test.go
Show resolved Hide resolved session/session_test.go
Show resolved Hide resolved store/tikv/2pc.go Outdated
Show resolved Hide resolved store/tikv/2pc.go Outdated

coocood added some commits Apr 30, 2019

@tiancaiamao

This comment has been minimized.

Copy link
Contributor

commented Apr 30, 2019

LGTM

Show resolved Hide resolved executor/adapter.go Outdated
Show resolved Hide resolved executor/adapter.go Outdated
Show resolved Hide resolved executor/adapter.go Outdated
Show resolved Hide resolved executor/adapter.go Outdated
Show resolved Hide resolved executor/adapter.go
Show resolved Hide resolved executor/prepared.go Outdated
Show resolved Hide resolved store/tikv/2pc.go
Show resolved Hide resolved executor/point_get.go
@coocood

This comment has been minimized.

Copy link
Member Author

commented May 6, 2019

@zhouqiang-cl

This comment has been minimized.

Copy link
Member

commented May 6, 2019

The make dev test takes too long and seems something wrong in this pr

@@ -130,7 +160,9 @@ func newTwoPhaseCommitter(txn *tikvTxn, connID uint64) (*twoPhaseCommitter, erro
}
delCnt++
}
keys = append(keys, k)
if isPessimistic && !bytes.Equal(k, c.primaryKey) {

This comment has been minimized.

Copy link
@lysu

lysu May 6, 2019

Member

maybe should be?

Suggested change
if isPessimistic && !bytes.Equal(k, c.primaryKey) {
if !isPessimistic || !bytes.Equal(k, c.primaryKey) {

This comment has been minimized.

Copy link
@lysu

lysu May 6, 2019

Member

after dig, maybe old one is ok too, c.primaryKey always be null if isPessimistic=true

This comment has been minimized.

Copy link
@jackysp

This comment has been minimized.

Copy link
@coocood

coocood May 9, 2019

Author Member

@lysu I know, but add isPessimistic is more explicit that this code is only related to pessimistic lock.

@lysu

This comment has been minimized.

Copy link
Member

commented May 6, 2019

rest LGTM for pessimistic disabled logic

Show resolved Hide resolved store/tikv/txn.go
}

for _, pair := range txn.assertions {
mutation, ok := mutations[string(pair.key)]
if !ok {
logutil.Logger(context.Background()).Error("ASSERTION FAIL!!! assertion exists but no mutation?",
zap.Stringer("assertion", pair))
// It's possible when a transaction inserted a key then deleted it later.

This comment has been minimized.

Copy link
@jackysp

jackysp May 9, 2019

Member

@tiancaiamao is this reasonable?

@jackysp
Copy link
Member

left a comment

rest LGTM

coocood added some commits May 10, 2019

Merge commit '5617809e922e2af62c14899429f69d5a267bec9d' into pessimis…
…tic-txn

Conflicts:
	go.mod
	go.sum
	store/mockstore/mocktikv/mvcc_leveldb.go
@zhouqiang-cl

This comment has been minimized.

Copy link
Member

commented May 10, 2019

make check always hang for very long time

coocood added some commits May 10, 2019

@coocood

This comment has been minimized.

Copy link
Member Author

commented May 10, 2019

/run-all-tests

@jackysp jackysp merged commit 373748a into pingcap:master May 11, 2019

14 checks passed

ci/circleci Your tests passed on CircleCI!
Details
idc-jenkins-ci-tidb/build Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/build_check_race Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/check_dev Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/check_dev_2 Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/common-test job succeeded
Details
idc-jenkins-ci-tidb/integration-common-test Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/integration-compatibility-test Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/integration-ddl-test Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/mybatis-test job succeeded
Details
idc-jenkins-ci-tidb/sqllogic-test-1 Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/sqllogic-test-2 Jenkins job succeeded.
Details
idc-jenkins-ci-tidb/unit-test Jenkins job succeeded.
Details
license/cla Contributor License Agreement is signed.
Details

@coocood coocood deleted the coocood:pessimistic-txn branch May 13, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.