Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

table: use evalBuffer to improve performance of lo ... (#18818) #19649

Merged
merged 3 commits into from
Sep 3, 2020

Conversation

ti-srebot
Copy link
Contributor

cherry-pick #18818 to release-4.0


What problem does this PR solve?

Issue Number: close #16667

Problem Summary: MutRowFromDatums is so heavy that cause locatePartition performance worse than expected. Especially in LOAD DATA, partition table need nearly twice time than general table.

What is changed and how it works?

What's Changed:

  • For single column partition key, like partition by hash(col) or partition by range(col), get integer value directly.
  • For expression as partition key, use evalBuffer to avoid MutRowFromDatums repeatly.

How it Works:
This two method will avoid using MutRowFromDatums many times. improve the write performance of partition table.

Related changes

  • Need to cherry-pick to the release branch

Check List

Tests

  • Unit test (will be added later)
  • Manual test (add detailed scripts or steps below)

Side effects

  • None

Release note

  • table: improve the write performance of partition table

Signed-off-by: ti-srebot <ti-srebot@pingcap.com>
@ti-srebot
Copy link
Contributor Author

/run-all-tests

@ti-srebot
Copy link
Contributor Author

@imtbkcat please accept the invitation then you can push to the cherry-pick pull requests.
https://github.com/ti-srebot/tidb/invitations

Copy link
Member

@zz-jason zz-jason left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-srebot ti-srebot added the status/LGT1 Indicates that a PR has LGTM 1. label Sep 1, 2020
@zz-jason
Copy link
Member

zz-jason commented Sep 1, 2020

/run-all-tests

@tiancaiamao
Copy link
Contributor

LGTM

@ti-srebot ti-srebot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Sep 3, 2020
@ti-srebot ti-srebot added status/LGT3 The PR has already had 3 LGTM. and removed status/LGT2 Indicates that a PR has LGTM 2. labels Sep 3, 2020
@jackysp
Copy link
Member

jackysp commented Sep 3, 2020

/merge

@ti-srebot ti-srebot added the status/can-merge Indicates a PR has been approved by a committer. label Sep 3, 2020
@ti-srebot
Copy link
Contributor Author

Your auto merge job has been accepted, waiting for:

  • 18164

@ti-srebot
Copy link
Contributor Author

/run-all-tests

@ti-srebot
Copy link
Contributor Author

@ti-srebot merge failed.

@zz-jason zz-jason changed the title table: use evalBuffer to improve performance of locatePartition (#18818) table: use evalBuffer to improve performance of lo ... (#18818) Sep 3, 2020
@zz-jason zz-jason merged commit 104d372 into pingcap:release-4.0 Sep 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
sig/transaction SIG:Transaction status/can-merge Indicates a PR has been approved by a committer. status/LGT3 The PR has already had 3 LGTM. type/enhancement type/4.0-cherry-pick
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants