db_stress begin tracking expected state after verification #9470
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously we enabled tracking expected state changes during `FinishInitDb()`, as soon as the DB was opened. This meant tracing was enabled during `VerifyDb()`. This cost extra CPU by requiring `DBImpl::trace_mutex_` to be acquired on each read operation. It was unnecessary since we know there are no expected state changes during the `VerifyDb()` phase. So, this PR delays tracking expected state changes until after the `VerifyDb()` phase has completed. Test Plan: Measured this PR reduced `VerifyDb()` 76% (387 -> 92 seconds) with `-disable_wal=1` (i.e., expected state tracking enabled). - benchmark command: `./db_stress -max_key=100000000 -ops_per_thread=1 -destroy_db_initially=1 -expected_values_dir=/dev/shm/dbstress_expected/ -db=/dev/shm/dbstress/ --clear_column_family_one_in=0 --disable_wal=1 --reopen=0` - without this PR, `VerifyDb()` takes 387 seconds: ``` 2022/01/30-21:43:04 Initializing worker threads Crash-recovery verification passed :) 2022/01/30-21:49:31 Starting database operations ``` - with this PR, `VerifyDb()` takes 92 seconds ``` 2022/01/30-21:59:06 Initializing worker threads Crash-recovery verification passed :) 2022/01/30-22:00:38 Starting database operations ```
|
@ajkr has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
There is an option UPDATE: |
|
Sorry let me update the description. I only meant to make the |
|
@ajkr has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
|
Thanks for the review! |
ajkr
added a commit
to ajkr/rocksdb
that referenced
this pull request
Jan 31, 2022
Despite attempts to optimize `db_stress` setup phase (i.e., pre-`OperateDb()`) latency in facebook#9470 and facebook#9475, it still always took tens of seconds. Since we still aren't able to setup a 100M key `db_stress` efficiently, we should reduce the number of keys. This PR reduces it 4x while increasing `value_size_mult` 4x (from its default value of 8) so that memtables and SST files fill up similarly quickly. Also disabled bzip2 compression since we'll probably never use it and I noticed many CI runs spending majority of CPU on bzip2 decompression.
facebook-github-bot
pushed a commit
that referenced
this pull request
Jan 31, 2022
Summary: Despite attempts to optimize `db_stress` setup phase (i.e., pre-`OperateDb()`) latency in #9470 and #9475, it still always took tens of seconds. Since we still aren't able to setup a 100M key `db_stress` quickly, we should reduce the number of keys. This PR reduces it 4x while increasing `value_size_mult` 4x (from its default value of 8) so that memtables and SST files fill at a similar rate compared to before this PR. Also disabled bzip2 compression since we'll probably never use it and I noticed many CI runs spending majority of CPU on bzip2 decompression. Pull Request resolved: #9476 Reviewed By: siying Differential Revision: D33898520 Pulled By: ajkr fbshipit-source-id: 855021784ad9664f2be5bce21f0339a1cf93230d
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously we enabled tracking expected state changes during
FinishInitDb(), as soon as the DB was opened. This meant tracing wasenabled during the
VerifyDb()in the setup phase (pre-OperateDb()). This cost extra CPU by requiringDBImpl::trace_mutex_to be acquired on each read operation. It wasunnecessary since we know there are no expected state changes during the
setup phase. So, this PR delays tracking expected state changes
until the end of the setup phase.
Test Plan:
Measured this PR reduced the
VerifyDb()time in the setup phase 76% (387 -> 92 seconds) with-disable_wal=1(i.e., expected state tracking enabled)../db_stress -max_key=100000000 -ops_per_thread=1 -destroy_db_initially=1 -expected_values_dir=/dev/shm/dbstress_expected/ -db=/dev/shm/dbstress/ --clear_column_family_one_in=0 --disable_wal=1 --reopen=0VerifyDb()in the setup phase takes 387 seconds:VerifyDb()in the setup phase takes 92 seconds