Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

import: enable auto compactions in import mode (#4086) #4199

Merged
merged 1 commit into from Feb 13, 2019

Conversation

@huachaohuang
Copy link
Contributor

commented Feb 12, 2019

Please refer to the original PR for more information: #4086

import: enable auto compactions in import mode (#4086)
Disabling auto compactions in import mode leads to some problems:
1. a lot of files will be accumulated in level 0, which slows down the
   sst ingestion
2. if a tikv restarts accidentally, it will stall for a long time
3. if we forget to switch back to normal mode after importing, tikv
   will perform very bad

Enabling auto compactions also simplifies the import process because
we don't need to compact every batch. According to previous tests,
compactions are not the bottleneck during importing data with
tidb-lightning.

All in all, I think disabling auto compactions in import mode does
more harm than good.

Signed-off-by: Huachao Huang <huachao.huang@gmail.com>

@huachaohuang huachaohuang requested a review from DorianZheng Feb 12, 2019

@huachaohuang huachaohuang requested a review from overvenus Feb 12, 2019

@DorianZheng
Copy link
Contributor

left a comment

LGTM

@huachaohuang

This comment has been minimized.

Copy link
Contributor Author

commented Feb 12, 2019

This will be released in 2.1.4, please update tidb-lightning accordingly.
/cc @lonng @kennytm

@kennytm

This comment has been minimized.

Copy link
Contributor

commented Feb 12, 2019

@huachaohuang Oh. The change is currently in pingcap/tidb-lightning#119.

@overvenus
Copy link
Contributor

left a comment

LGTM

@overvenus overvenus merged commit 43300ed into tikv:release-2.1 Feb 13, 2019

2 checks passed

DCO All commits are signed off!
Details
idc-jenkins-ci/test Jenkins job succeeded.
Details

@huachaohuang huachaohuang deleted the huachaohuang:release-2.1 branch Feb 13, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.