cover single level universal in crash test #6818
Conversation
There was a problem hiding this comment.
@ajkr has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
ajkr
force-pushed
the
crash-test-single-level-universal
branch
from
72053ff
to
126fddd
Compare
| @@ -186,6 +186,9 @@ DEFINE_int64(compressed_cache_size, -1, | |||
| DEFINE_int32(compaction_style, ROCKSDB_NAMESPACE::Options().compaction_style, | |||
| ""); | |||
|
|
|||
| DEFINE_int32(num_levels, ROCKSDB_NAMESPACE::Options().num_levels, | |||
There was a problem hiding this comment.
Maybe add this to db_stress_common.h as well.
Test Plan: fast whitebox test and verify there are some single-level universal and some multi-level universal runs. ``` $ python ./tools/db_crashtest.py whitebox --simple -max_key=1000000 -value_size_mult=33 -write_buffer_size=524288 -target_file_size_base=524288 -max_bytes_for_level_base=2097152 --duration=120 --interval=10 --ops_per_thread=1000 --random_kill_odd=887 ```
ajkr
force-pushed
the
crash-test-single-level-universal
branch
from
126fddd
to
1d2c622
Compare
|
@ajkr has updated the pull request. Re-import the pull request |
| @@ -186,6 +186,9 @@ DEFINE_int64(compressed_cache_size, -1, | |||
| DEFINE_int32(compaction_style, ROCKSDB_NAMESPACE::Options().compaction_style, | |||
| ""); | |||
|
|
|||
| DEFINE_int32(num_levels, ROCKSDB_NAMESPACE::Options().num_levels, | |||
There was a problem hiding this comment.
@ajkr has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.
| # it sometimes. | ||
| if random.randint(0, 1) == 1: | ||
| additional_opts.update({ | ||
| "num_levels": 1, |
There was a problem hiding this comment.
Do we need to migrate the data if there is existing data in non-L0 levels for this case?
There was a problem hiding this comment.
huh, I assumed that happened automatically, as in H2 of the whitebox crash test, it alternates between universal and FIFO compaction. So I thought it'd have to clear the DB between runs...
There was a problem hiding this comment.
How about set destroy_db_initially to true for these cases that may change compaction style? At least until we come up with something better than cycling through the rare compaction styles during the second half of crash test...
Summary: An add-on to #6818 to complete adding single-level universal compaction to stress/crash testing. Pull Request resolved: #10732 Test Plan: - Locally run for 10 min `python3 ./tools/db_crashtest.py whitebox --simple --compaction_style=1 --num_levels=1 -max_key=1000000 -value_size_mult=33 -write_buffer_size=524288 -target_file_size_base=524288 -max_bytes_for_level_base=2097152 --duration=120 --interval=10 --ops_per_thread=1000 --random_kill_odd=887` - Check LOG to confirm single-level universal compaction is called - Manual testing and log checking to ensure destroy_db_initially=1 is correctly set across runs with different compaction styles (i.e, in the second half of whitebox testing). - [ongoing]CI jobs stress test Reviewed By: ajkr Differential Revision: D39797612 Pulled By: ajkr fbshipit-source-id: 16f5c40c3464c57360c06c8305f92118e426149c
Test Plan:
fast whitebox test and verify there are some single-level universal and
some multi-level universal runs.