Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set blob_run_mode=ReadOnly for non-default CFs #4991

Merged
merged 4 commits into from Jul 1, 2019

Conversation

Projects
None yet
4 participants
@yiwu-arbug
Copy link
Contributor

commented Jun 27, 2019

What have you changed? (mandatory)

Currently titan is disabled for non-default CFs by setting min_blob_size=4GB. Changing to set blob_run_mode=ReadOnly which will disable titan even value size is >4GB (though it will never happen)

What are the type of the changes? (mandatory)

Improvement

How has this PR been tested? (mandatory)

Manually start tikv-server and check rocksdb log to set blob_run_mode is set.

Does this PR affect documentation (docs) or release note? (mandatory)

no

Does this PR affect tidb-ansible update? (mandatory)

no

Refer to a related PR or issue link (optional)

Benchmark result if necessary (optional)

Add a few positive/negative examples (optional)

yiwu-arbug added some commits Jun 27, 2019

Set blob_run_mode=read_only for non-default CFs
Signed-off-by: Yi Wu <yiwu@pingcap.com>
add missing set_blob_run_mode
Signed-off-by: Yi Wu <yiwu@pingcap.com>
@Connor1996
Copy link
Member

left a comment

LGTM

@Connor1996

This comment has been minimized.

Copy link
Member

commented Jun 28, 2019

Please fix the config test

fix test
Signed-off-by: Yi Wu <yiwu@pingcap.com>
@@ -279,15 +279,15 @@ fn test_serde_custom_tikv_config() {
soft_pending_compaction_bytes_limit: ReadableSize::gb(12),
hard_pending_compaction_bytes_limit: ReadableSize::gb(12),
titan: TitanCfConfig {
min_blob_size: ReadableSize::gb(4).0 as u64, // disable titan default
min_blob_size: 1024, // default value

This comment has been minimized.

Copy link
@Connor1996

Connor1996 Jun 28, 2019

Member

Please use non-default value in test

This comment has been minimized.

Copy link
@yiwu-arbug

yiwu-arbug Jun 28, 2019

Author Contributor

This is for non-default CFs. The min_blob_size here reflect default value since we don't set their value from test-custom.toml. And it doesn't make sense to set them.

@zhangjinpeng1987
Copy link
Member

left a comment

LGTM

@yiwu-arbug

This comment has been minimized.

Copy link
Contributor Author

commented Jun 30, 2019

@Connor1996 any more comments? thanks.

@siddontang

This comment has been minimized.

Copy link
Contributor

commented Jul 1, 2019

can we change it dynamically later?

@Connor1996

This comment has been minimized.

Copy link
Member

commented Jul 1, 2019

@siddontang it can be changed dynamically.

@Connor1996
Copy link
Member

left a comment

LGTM

@Connor1996 Connor1996 merged commit a0f8be8 into tikv:master Jul 1, 2019

2 checks passed

DCO All commits are signed off!
Details
idc-jenkins-ci/test Jenkins job succeeded.
Details

mapleFU pushed a commit to mapleFU/tikv that referenced this pull request Jul 1, 2019

Set blob_run_mode=ReadOnly for non-default CFs (tikv#4991)
* Set blob_run_mode=read_only for non-default CFs

Signed-off-by: Yi Wu <yiwu@pingcap.com>
Signed-off-by: mapleFU <1506118561@qq.com>

mapleFU pushed a commit to mapleFU/tikv that referenced this pull request Jul 1, 2019

Set blob_run_mode=ReadOnly for non-default CFs (tikv#4991)
* Set blob_run_mode=read_only for non-default CFs

Signed-off-by: Yi Wu <yiwu@pingcap.com>

mapleFU added a commit to mapleFU/tikv that referenced this pull request Jul 1, 2019

Set blob_run_mode=ReadOnly for non-default CFs (tikv#4991)
* Set blob_run_mode=read_only for non-default CFs

Signed-off-by: yiwu-arbug <yiwu@pingcap.com>

mapleFU added a commit to mapleFU/tikv that referenced this pull request Jul 1, 2019

Set blob_run_mode=ReadOnly for non-default CFs (tikv#4991)
* Set blob_run_mode=read_only for non-default CFs

Signed-off-by: Yi Wu <yiwu@pingcap.com>

mapleFU added a commit to mapleFU/tikv that referenced this pull request Jul 1, 2019

Set blob_run_mode=ReadOnly for non-default CFs (tikv#4991)
* Set blob_run_mode=read_only for non-default CFs

Signed-off-by: Yi Wu <yiwu@pingcap.com>

mapleFU added a commit to mapleFU/tikv that referenced this pull request Jul 1, 2019

Set blob_run_mode=ReadOnly for non-default CFs (tikv#4991)
* Set blob_run_mode=read_only for non-default CFs

Signed-off-by: yiwu-arbug <yiwu@pingcap.com>

@yiwu-arbug yiwu-arbug deleted the yiwu-arbug:createcf_mode branch Jul 1, 2019

Connor1996 added a commit to Connor1996/tikv that referenced this pull request Jul 15, 2019

Set blob_run_mode=ReadOnly for non-default CFs (tikv#4991)
* Set blob_run_mode=read_only for non-default CFs

Signed-off-by: Yi Wu <yiwu@pingcap.com>

Connor1996 added a commit to Connor1996/tikv that referenced this pull request Jul 15, 2019

Set blob_run_mode=ReadOnly for non-default CFs (tikv#4991)
* Set blob_run_mode=read_only for non-default CFs

Signed-off-by: Yi Wu <yiwu@pingcap.com>
Signed-off-by: Connor1996 <zbk602423539@gmail.com>

Connor1996 added a commit to Connor1996/tikv that referenced this pull request Jul 15, 2019

Set blob_run_mode=ReadOnly for non-default CFs (tikv#4991)
* Set blob_run_mode=read_only for non-default CFs

Signed-off-by: Yi Wu <yiwu@pingcap.com>
Signed-off-by: Connor1996 <zbk602423539@gmail.com>

Connor1996 added a commit to Connor1996/tikv that referenced this pull request Jul 15, 2019

Set blob_run_mode=ReadOnly for non-default CFs (tikv#4991)
* Set blob_run_mode=read_only for non-default CFs

Signed-off-by: Yi Wu <yiwu@pingcap.com>
Signed-off-by: Connor1996 <zbk602423539@gmail.com>

youjiali1995 added a commit that referenced this pull request Jul 15, 2019

Cherry-pick some Titan related commits (#5084)
* count size of blob files in used size (#5060)

Signed-off-by: Connor1996 <zbk602423539@gmail.com>

* Check max required open files for titan (#5026)

* Check max required open files for titan

Signed-off-by: Xiaoguang Sun <sunxiaoguang@zhihu.com>

* Add comment to new change

Signed-off-by: Xiaoguang Sun <sunxiaoguang@zhihu.com>
Signed-off-by: Connor1996 <zbk602423539@gmail.com>

* Add blob-run-mode for titan (#4865)

* add blob-run-mode for titan

Signed-off-by: tabokie <xy.tao@outlook.com>

* address comment

Signed-off-by: tabokie <xy.tao@outlook.com>

* fix config test

Signed-off-by: tabokie <xy.tao@outlook.com>
Signed-off-by: Connor1996 <zbk602423539@gmail.com>

* Set blob_run_mode=ReadOnly for non-default CFs (#4991)

* Set blob_run_mode=read_only for non-default CFs

Signed-off-by: Yi Wu <yiwu@pingcap.com>
Signed-off-by: Connor1996 <zbk602423539@gmail.com>

* engine: add titan metrics (#4772)

* add titan metrics

Signed-off-by: Connor1996 <zbk602423539@gmail.com>

* metrics: add titan metrics (#4836)

* Add titan blob-related metrics

Signed-off-by: tabokie <xy.tao@outlook.com>

* fmt

Signed-off-by: tabokie <xy.tao@outlook.com>

* reset cargo

Signed-off-by: tabokie <xy.tao@outlook.com>
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.