New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

config: add wal bytes per sync #2407

Merged
merged 5 commits into from Oct 24, 2017

Conversation

Projects
None yet
3 participants
@siddontang
Contributor

siddontang commented Oct 23, 2017

Use ycsb tidb-ycsb -concurrency 200 -workload=F -duration=10m raw://172.16.20.1:2379, we can find that usingwal_bytes_per_sync can improve the 3% QPS if sync = false, but have no big effect if sync = true.

We need to do more benchmark to ensure a suitable value, so here I just add the configuration.
For bytes_per_sync, we can use rate limiter which sets bytes_per_sync to 1 MB by default.

Raft 0 MB Raft 64 KB Raft 1 MB
KV 1 MB + Sync false 32038.3 32880.7 32938.9
KV 0 MB + Sync false 31533.7 32876.3 32742.1
KV 0 MB + Sync true 25017.8 24993.2 24492.9

/cc @zhangjinpeng1987 @huachaohuang

@huachaohuang

This comment has been minimized.

Show comment
Hide comment
@huachaohuang

huachaohuang Oct 23, 2017

Member

You need to update the etc template and the test template too.

Member

huachaohuang commented Oct 23, 2017

You need to update the etc template and the test template too.

@siddontang

This comment has been minimized.

Show comment
Hide comment
@siddontang
Contributor

siddontang commented Oct 23, 2017

@zhangjinpeng1987

This comment has been minimized.

Show comment
Hide comment
@zhangjinpeng1987
Member

zhangjinpeng1987 commented Oct 23, 2017

LGTM

@siddontang

This comment has been minimized.

Show comment
Hide comment
@siddontang
Contributor

siddontang commented Oct 24, 2017

Show outdated Hide outdated etc/config-template.toml Outdated
@siddontang

This comment has been minimized.

Show comment
Hide comment
@siddontang
Contributor

siddontang commented Oct 24, 2017

@huachaohuang

LGTM

@huachaohuang

This comment has been minimized.

Show comment
Hide comment
@huachaohuang

huachaohuang Oct 24, 2017

Member

/run-integration-tests

Member

huachaohuang commented Oct 24, 2017

/run-integration-tests

@siddontang siddontang merged commit 716633c into tikv:master Oct 24, 2017

6 checks passed

ci/circleci Your tests passed on CircleCI!
Details
jenkins-ci-tikv/build Jenkins job succeeded.
Details
jenkins-ci-tikv/integration-common-test Jenkins job succeeded.
Details
jenkins-ci-tikv/integration-compatibility-test Jenkins job succeeded.
Details
jenkins-ci-tikv/integration-ddl-test Jenkins job succeeded.
Details
license/cla Contributor License Agreement is signed.
Details

@siddontang siddontang deleted the siddontang:siddontang/sync-opt branch Oct 24, 2017

huachaohuang added a commit to huachaohuang/tikv that referenced this pull request Nov 10, 2017

huachaohuang added a commit that referenced this pull request Nov 10, 2017

Cherry pick commits to release-1.0 (#2466)
* config: add wal bytes per sync (#2407)

* Reset RocksDB statistics periodically (#2443)

* coprocessor: fix do_div_mod bug (#2456)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment