Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

store/tikv: fix duplicated lock keys caused data inconsistency. #16752

Merged
merged 5 commits into from Apr 23, 2020

Conversation

coocood
Copy link
Member

@coocood coocood commented Apr 23, 2020

What problem does this PR solve?

Problem Summary:

  • Update MultipleTable field is not correct, adding SelectLock operator for multiple table update causes LockKeys contains duplicated keys.

  • When LockKeys contains duplicated keys, the transaction may overwrite Put operation with Lock operation, cause admin check table failure.

What is changed and how it works?

What's Changed:

  • Check update single table by TableRefs.Right == nil
  • Deduplicate keys in LockKeys method.

Related changes

  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test

Side effects

  • Performance regression
    • Consumes more CPU

Release note

fix duplicated lock keys caused data inconsistency.

@coocood coocood added type/bug-fix This PR fixes a bug. sig/transaction SIG:Transaction needs-cherry-pick-4.0 labels Apr 23, 2020
@coocood coocood requested a review from a team as a code owner April 23, 2020 06:58
@ghost ghost requested review from winoros and removed request for a team April 23, 2020 06:58
Copy link
Contributor

@cfzjywxk cfzjywxk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@@ -3259,7 +3259,7 @@ func (b *PlanBuilder) buildUpdate(ctx context.Context, update *ast.UpdateStmt) (
}
}
if b.ctx.GetSessionVars().TxnCtx.IsPessimistic {
if !update.MultipleTable {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe remove these unused fields to avoid using mistakenly in the future, so dangerous

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's in another repo, we can do it later.

})
deduped := keys[:1]
for i := 1; i < len(keys); i++ {
if !bytes.Equal(deduped[len(deduped)-1], keys[i]) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add warn logs here to remind there could be possible bugs?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we need to support lock duplicated keys in the future.

@@ -3259,7 +3259,7 @@ func (b *PlanBuilder) buildUpdate(ctx context.Context, update *ast.UpdateStmt) (
}
}
if b.ctx.GetSessionVars().TxnCtx.IsPessimistic {
if !update.MultipleTable {
if update.TableRefs.TableRefs.Right == nil {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is better to add some comments.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And does

if !delete.IsMultiTable {

work well?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to check this

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it works as expected.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The world is too dangerous...

@codecov
Copy link

codecov bot commented Apr 23, 2020

Codecov Report

Merging #16752 into master will not change coverage.
The diff coverage is n/a.

@@             Coverage Diff             @@
##             master     #16752   +/-   ##
===========================================
  Coverage   80.4432%   80.4432%           
===========================================
  Files           507        507           
  Lines        137477     137477           
===========================================
  Hits         110591     110591           
  Misses        18282      18282           
  Partials       8604       8604           

@@ -357,6 +359,7 @@ func (txn *tikvTxn) LockKeys(ctx context.Context, lockCtx *kv.LockCtx, keysInput
if len(keys) == 0 {
return nil
}
keys = deduplicateKeys(keys)
Copy link
Member

@jackysp jackysp Apr 23, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's the second line of defense.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But it hurts the performance.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the proportion would be tiny.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok

Copy link
Member

@jackysp jackysp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jackysp
Copy link
Member

jackysp commented Apr 23, 2020

/merge

@sre-bot sre-bot added the status/can-merge Indicates a PR has been approved by a committer. label Apr 23, 2020
@sre-bot
Copy link
Contributor

sre-bot commented Apr 23, 2020

/run-all-tests

@sre-bot
Copy link
Contributor

sre-bot commented Apr 23, 2020

@coocood merge failed.

@bb7133 bb7133 added the priority/release-blocker This PR blocks a release. Please review it ASAP. label Apr 23, 2020
@coocood
Copy link
Member Author

coocood commented Apr 23, 2020

/run-integration-copr-test

@coocood coocood merged commit c32fc6d into pingcap:master Apr 23, 2020
@coocood coocood deleted the fix-dup-lock-keys branch April 23, 2020 10:46
sre-bot pushed a commit to sre-bot/tidb that referenced this pull request Apr 23, 2020
Signed-off-by: sre-bot <sre-bot@pingcap.com>
@sre-bot
Copy link
Contributor

sre-bot commented Apr 23, 2020

cherry pick to release-4.0 in PR #16769

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority/release-blocker This PR blocks a release. Please review it ASAP. sig/transaction SIG:Transaction status/can-merge Indicates a PR has been approved by a committer. type/bug-fix This PR fixes a bug.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants