Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

raftstore: optimize write priority scheduling path #14335

Merged
merged 10 commits into from Mar 14, 2023
Merged

raftstore: optimize write priority scheduling path #14335

merged 10 commits into from Mar 14, 2023

Conversation

Connor1996
Copy link
Member

@Connor1996 Connor1996 commented Mar 3, 2023 •
edited

What is changed and how it works?

Issue Number: Ref #14353

Optimize write priority scheduling path including:
- replace `DashMap` with `RwLock` in the resource controller 
- use visit pattern for consuming msg resource to avoid constructing hashmap
- introduce `ParsedEntry` to avoid parsing raft command from entry data repeatedly in different places

Check List

Tests

  • Unit test
  • Integration test

Have 1% improvement with resource control enabled, compared to master.
image

Release note 

optimize write priority scheduling path performance overhead

@Connor1996
optimize write priority scheduling path
c5f38e1 
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
@ti-chi-bot
Copy link
Member

ti-chi-bot commented Mar 3, 2023 •
edited

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • glorv
  • nolouch

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@Connor1996
Copy link
Member Author

/release

@Connor1996
fix
ee5997c 
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
@Connor1996
Copy link
Member Author

/release

@sre-bot
Copy link
Contributor

sre-bot commented Mar 3, 2023

download tikv at http://fileserver.pingcap.net/download/builds/pingcap/tikv/pr/c5f38e1d3abdfc8e1e6d68ee9ad767850162db02/centos7/tikv-server.tar.gz

@sre-bot
Copy link
Contributor

sre-bot commented Mar 3, 2023

download tikv at http://fileserver.pingcap.net/download/builds/pingcap/tikv/pr/ee5997c47c73b74f36f20ff00bbea0fab2b14323/centos7/tikv-server.tar.gz

@Connor1996
fix bug
eac1d0a 
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
@Connor1996
Copy link
Member Author

/release

@sre-bot
Copy link
Contributor

sre-bot commented Mar 6, 2023

download tikv at http://fileserver.pingcap.net/download/builds/pingcap/tikv/pr/eac1d0aaffca563bbbf8a193164a6bb5129beff3/centos7/tikv-server.tar.gz

@Connor1996
add test
8d83e81 
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
@Connor1996
clean
d7ddbdc 
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
@Connor1996
fix test
a4eb4e6 
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
@Connor1996
Copy link
Member Author

/test

@Connor1996 Connor1996 marked this pull request as ready for review March 7, 2023 07:16
nolouch
nolouch reviewed Mar 7, 2023
View reviewed changes
components/raftstore/src/store/util.rs
// ParsedEntry wraps raft-proto `Entry` and used to avoid parsing raft command
// from entry's data repeatedly. The parsed command may be used in multiple
// places, so cache it at the first place.
pub struct ParsedEntry {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consider a better name? maybe as use Entry , RaftEntry or LogEntry directly?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These names can't be distinguished with the current raftpb::Entry

Copy link
Contributor

@nolouch nolouch Mar 13, 2023 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a little strange with parsed and there has parse function.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe call it LazyEntry?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Parsed describes the behavior, that it provides parsed items (cmd and conf_change) to outers. parse is just an inner function.

Copy link
Contributor

@nolouch nolouch Mar 14, 2023 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about PlainEntry or DecodedCmdEntry?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Entry is defined in proto, I can't change the name.

@Connor1996
Copy link
Member Author

/test

glorv
glorv approved these changes Mar 13, 2023
View reviewed changes
Copy link
Contributor

@glorv glorv left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

components/raftstore/src/store/entry_storage.rs Outdated Show resolved Hide resolved
@ti-chi-bot ti-chi-bot added the status/LGT1 Status: PR - There is already 1 approval label Mar 13, 2023
@Connor1996
address comment
fa34db9 
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
nolouch
nolouch approved these changes Mar 13, 2023
View reviewed changes
Copy link
Contributor

@nolouch nolouch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ti-chi-bot ti-chi-bot added status/LGT2 Status: PR - There are already 2 approvals and removed status/LGT1 Status: PR - There is already 1 approval labels Mar 13, 2023
@Connor1996
Copy link
Member Author

/test

@Connor1996
use same version
c75eb95 
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
@Connor1996
Copy link
Member Author

/merge

@ti-chi-bot
Copy link
Member

@Connor1996: It seems you want to merge this PR, I will help you trigger all the tests:

/run-all-tests

You only need to trigger /merge once, and if the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

If you have any questions about the PR merge process, please refer to pr process.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: c75eb95

@ti-chi-bot ti-chi-bot added the status/can-merge Status: Can merge to base branch label Mar 13, 2023
@ti-chi-bot
Copy link
Member

@Connor1996: Your PR was out of date, I have automatically updated it for you.

If the CI test fails, you just re-trigger the test that failed and the bot will merge the PR for you after the CI passes.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository.

@Connor1996
Copy link
Member Author

/test

@ti-chi-bot ti-chi-bot merged commit 852af46 into tikv:master Mar 14, 2023
1 check passed
@ti-chi-bot ti-chi-bot added this to the Pool milestone Mar 14, 2023
@glorv glorv mentioned this pull request Mar 16, 2023
Merged
Connor1996 added a commit to Connor1996/tikv that referenced this pull request Mar 17, 2023
@Connor1996
Revert "raftstore: optimize write priority scheduling path (tikv#14335)"
f9b7921 
This reverts commit 852af46.
Connor1996 added a commit to Connor1996/tikv that referenced this pull request Mar 17, 2023
@Connor1996
Revert "raftstore: optimize write priority scheduling path (tikv#14335)"
b8901f2 
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nolouch nolouch nolouch approved these changes

@glorv glorv glorv approved these changes

@tabokie tabokie tabokie approved these changes

Assignees
No one assigned
Labels
release-note size/XXL status/can-merge Status: Can merge to base branch status/LGT2 Status: PR - There are already 2 approvals
Projects
None yet
Milestone
Pool
Development

Successfully merging this pull request may close these issues.

None yet
6 participants
@Connor1996 @ti-chi-bot @sre-bot @glorv @nolouch @tabokie