Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

lightning: duplicate resolution support to use DELETE to cleanup clustered table #44799

Merged
merged 8 commits into from Jul 3, 2023
Merged

lightning: duplicate resolution support to use DELETE to cleanup clustered table #44799

merged 8 commits into from Jul 3, 2023

Conversation

lance6716
Copy link
Contributor

@lance6716 lance6716 commented Jun 19, 2023 •
edited

What problem does this PR solve?

Issue Number: ref #44723

Problem Summary:

What is changed and how it works?

The fix is made at TiKV side tikv/tikv#14960

this PR fix a bug that clustered PK will generate non-clustered PK index KV and be deleted when resolve duplication

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@lance6716
lightning: duplicate resolution support to use DELETE to cleanup
a47961f 
Signed-off-by: lance6716 <lance6716@gmail.com>
@ti-chi-bot ti-chi-bot bot added release-note-none size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jun 19, 2023
@lance6716
Copy link
Contributor Author

/run-integration-br-test

@lance6716 lance6716 changed the title lightning: duplicate resolution support to use DELETE to cleanup [WIP]lightning: duplicate resolution support to use DELETE to cleanup Jun 19, 2023
@ti-chi-bot ti-chi-bot bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 19, 2023
@lance6716
fix CI
b6355f8 
Signed-off-by: lance6716 <lance6716@gmail.com>
@lance6716
Copy link
Contributor Author

/run-integration-br-test

@lance6716
fix CI
496186b 
Signed-off-by: lance6716 <lance6716@gmail.com>
@ti-chi-bot ti-chi-bot bot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Jun 20, 2023
@lance6716 lance6716 changed the title [WIP]lightning: duplicate resolution support to use DELETE to cleanup lightning: duplicate resolution support to use DELETE to cleanup clustered table Jun 20, 2023
@ti-chi-bot ti-chi-bot bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jun 20, 2023
@lance6716
uncomment
e720391 
Signed-off-by: lance6716 <lance6716@gmail.com>
@lance6716
fix bazel_prepare
cf26eda 
Signed-off-by: lance6716 <lance6716@gmail.com>
@lance6716
Merge branch 'master' of github.com:pingcap/tidb into delete-dedup
5da04a9
@hawkingrei
Copy link
Member

/test tiprow_fast_test

@ti-chi-bot
Copy link

ti-chi-bot bot commented Jun 25, 2023

@hawkingrei: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

  • /test build
  • /test canary-scan-security
  • /test check-dev
  • /test check-dev2
  • /test mysql-test
  • /test pingcap/tidb/canary_ghpr_unit_test
  • /test pull-integration-br-test
  • /test pull-integration-mysql-test
  • /test unit-test

Use /test all to run the following jobs that were automatically triggered:

  • pingcap/tidb/ghpr_build
  • pingcap/tidb/ghpr_check
  • pingcap/tidb/ghpr_check2
  • pingcap/tidb/ghpr_mysql_test
  • pingcap/tidb/ghpr_unit_test

In response to this:

/test tiprow_fast_test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@niubell niubell requested a review from D3Hunter June 29, 2023 08:59
D3Hunter
D3Hunter reviewed Jul 3, 2023
View reviewed changes
Copy link
Contributor

@D3Hunter D3Hunter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest lgtm

br/pkg/lightning/backend/kv/kv2sql.go

var buffer []types.Datum
var indexBuffer []byte
for _, index := range indices {
// skip clustered PK
if index.Meta().Primary && isCommonHandle {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we skip clustered & single-int column kind of pk?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added test 7338cc1

When PKIsHandle (test file line 83), I remember the PK will not appear in indices, so no need to skip it.

@lance6716
address comment
7338cc1 
Signed-off-by: lance6716 <lance6716@gmail.com>
@ti-chi-bot ti-chi-bot bot deleted a comment from ti-chi-bot Jul 3, 2023
@lance6716
fix bazel
5a28767 
Signed-off-by: lance6716 <lance6716@gmail.com>
@ti-chi-bot ti-chi-bot bot deleted a comment from ti-chi-bot Jul 3, 2023
GMHDBJD
GMHDBJD approved these changes Jul 3, 2023
View reviewed changes
Copy link
Contributor

@GMHDBJD GMHDBJD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot ti-chi-bot bot added the lgtm label Jul 3, 2023
@ti-chi-bot
Copy link

ti-chi-bot bot commented Jul 3, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: D3Hunter, GMHDBJD

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot
Copy link

ti-chi-bot bot commented Jul 3, 2023

[LGTM Timeline notifier]

Timeline:

  • 2023-07-03 04:04:12.538921828 +0000 UTC m=+3884.472555249: ☑️ agreed by D3Hunter.
  • 2023-07-03 08:40:41.42785807 +0000 UTC m=+20473.361491489: ☑️ agreed by GMHDBJD.

@lance6716
Copy link
Contributor Author

/retest

@lance6716
Copy link
Contributor Author

/run-integration-br-test

@tiprow
Copy link

tiprow bot commented Jul 3, 2023

@lance6716: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@lance6716
Copy link
Contributor Author

/ok-to-test

@ti-chi-bot ti-chi-bot bot added the ok-to-test label Jul 3, 2023
@ti-chi-bot ti-chi-bot bot deleted a comment from ti-chi-bot Jul 3, 2023
@ti-chi-bot ti-chi-bot bot merged commit 60784a3 into pingcap:master Jul 3, 2023
17 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@D3Hunter D3Hunter D3Hunter approved these changes

@GMHDBJD GMHDBJD GMHDBJD approved these changes

Assignees
No one assigned
Labels
approved lgtm ok-to-test release-note-none size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@lance6716 @hawkingrei @D3Hunter @GMHDBJD