Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check max required open files for titan #5026

Merged

Conversation

Projects
None yet
6 participants
@sunxiaoguang
Copy link
Collaborator

commented Jul 5, 2019

Signed-off-by: Xiaoguang Sun sunxiaoguang@zhihu.com

What have you changed? (mandatory)

Check max open files required by titan engine when it is enabled.

Please explain IN DETAIL what the changes are in this PR and why they are needed:

  • Summarize your change (mandatory)
    Add max open files required by titan engine when it is enabled

What are the type of the changes? (mandatory)

  • Misc (other changes)

How has this PR been tested? (mandatory)

Manually tested

Does this PR affect documentation (docs) or release note? (mandatory)

No

Does this PR affect tidb-ansible update? (mandatory)

No

Refer to a related PR or issue link (optional)

NA

Benchmark result if necessary (optional)

NA

Add a few positive/negative examples (optional)

NA

Check max required open files for titan
Signed-off-by: Xiaoguang Sun <sunxiaoguang@zhihu.com>
@sre-bot

This comment has been minimized.

Copy link
Collaborator

commented Jul 5, 2019

Hi contributor, thanks for your PR.

This patch needs to be approved by someone of admins. They should reply with "/ok-to-test" to accept this PR for running test automatically.

@@ -434,8 +434,12 @@ fn pre_start(cfg: &TiKvConfig) {
}

fn check_system_config(config: &TiKvConfig) {
let mut rocksdb_max_open_files = config.rocksdb.max_open_files;
if config.rocksdb.titan.enabled {
rocksdb_max_open_files *= 2;

This comment has been minimized.

Copy link
@siddontang

siddontang Jul 5, 2019

Contributor

please add a comment here

This comment has been minimized.

Copy link
@zhangjinpeng1987

zhangjinpeng1987 Jul 5, 2019

Member

Is it possible that Titan has more files than RocksDB @Connor1996

This comment has been minimized.

Copy link
@sunxiaoguang

sunxiaoguang Jul 5, 2019

Author Collaborator

Done.

please add a comment here

This comment has been minimized.

Copy link
@sunxiaoguang

sunxiaoguang Jul 5, 2019

Author Collaborator

Is it possible that Titan has more files than RocksDB @Connor1996

Looks like Titan uses the same max open files setup as rocksdb does.

image

@zhangjinpeng1987

This comment has been minimized.

Copy link
Member

commented Jul 5, 2019

Please sign-off

Add comment to new change
Signed-off-by: Xiaoguang Sun <sunxiaoguang@zhihu.com>

@sunxiaoguang sunxiaoguang force-pushed the sunxiaoguang:check_max_open_files_for_titan branch from 2c60f91 to 9750f50 Jul 5, 2019

@sunxiaoguang

This comment has been minimized.

Copy link
Collaborator Author

commented Jul 5, 2019

Please sign-off

Oops, It's there now.

@yiwu-arbug yiwu-arbug added the C: Titan label Jul 5, 2019

@Connor1996
Copy link
Member

left a comment

LGTM

@zhangjinpeng1987
Copy link
Member

left a comment

LGTM

@sunxiaoguang sunxiaoguang merged commit cd071db into tikv:master Jul 5, 2019

2 checks passed

DCO All commits are signed off!
Details
idc-jenkins-ci/test Jenkins job succeeded.
Details

@sunxiaoguang sunxiaoguang deleted the sunxiaoguang:check_max_open_files_for_titan branch Jul 9, 2019

Connor1996 added a commit to Connor1996/tikv that referenced this pull request Jul 15, 2019

Check max required open files for titan (tikv#5026)
* Check max required open files for titan

Signed-off-by: Xiaoguang Sun <sunxiaoguang@zhihu.com>

* Add comment to new change

Signed-off-by: Xiaoguang Sun <sunxiaoguang@zhihu.com>

Connor1996 added a commit to Connor1996/tikv that referenced this pull request Jul 15, 2019

Check max required open files for titan (tikv#5026)
* Check max required open files for titan

Signed-off-by: Xiaoguang Sun <sunxiaoguang@zhihu.com>

* Add comment to new change

Signed-off-by: Xiaoguang Sun <sunxiaoguang@zhihu.com>
Signed-off-by: Connor1996 <zbk602423539@gmail.com>

youjiali1995 added a commit that referenced this pull request Jul 15, 2019

Cherry-pick some Titan related commits (#5084)
* count size of blob files in used size (#5060)

Signed-off-by: Connor1996 <zbk602423539@gmail.com>

* Check max required open files for titan (#5026)

* Check max required open files for titan

Signed-off-by: Xiaoguang Sun <sunxiaoguang@zhihu.com>

* Add comment to new change

Signed-off-by: Xiaoguang Sun <sunxiaoguang@zhihu.com>
Signed-off-by: Connor1996 <zbk602423539@gmail.com>

* Add blob-run-mode for titan (#4865)

* add blob-run-mode for titan

Signed-off-by: tabokie <xy.tao@outlook.com>

* address comment

Signed-off-by: tabokie <xy.tao@outlook.com>

* fix config test

Signed-off-by: tabokie <xy.tao@outlook.com>
Signed-off-by: Connor1996 <zbk602423539@gmail.com>

* Set blob_run_mode=ReadOnly for non-default CFs (#4991)

* Set blob_run_mode=read_only for non-default CFs

Signed-off-by: Yi Wu <yiwu@pingcap.com>
Signed-off-by: Connor1996 <zbk602423539@gmail.com>

* engine: add titan metrics (#4772)

* add titan metrics

Signed-off-by: Connor1996 <zbk602423539@gmail.com>

* metrics: add titan metrics (#4836)

* Add titan blob-related metrics

Signed-off-by: tabokie <xy.tao@outlook.com>

* fmt

Signed-off-by: tabokie <xy.tao@outlook.com>

* reset cargo

Signed-off-by: tabokie <xy.tao@outlook.com>
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.