New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Resharding tool for sharded rocksdb #34667
Conversation
src/kv/RocksDBStore.cc
Outdated
keys_per_iterator ++; | ||
|
||
//check if need to write batch | ||
if (bytes_in_batch > 1000000 || |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should we make these dev options? could see potentially wanting to do it in larger batches for speed, or smaller to throttle things.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
looks good!
Created function that prepares environment for _open_db. Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Reshard ability allows to change current sharding schema, without change to data. Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Now merge operator name also includes operators from dedicated to shards. This makes it possible to reshard that will move prefix with merge operator away from default column family. Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
…ing resharding Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
…ansferred Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
Added possibility to control batch size and iterator refresh time for resharding process. Replaced getenv() with new control for resharding unittests. Signed-off-by: Adam Kupczyk <akupczyk@redhat.com>
47beff3
to
38ac896
Compare
retest this please |
Introduces reshard() method to RocksDBStore.
It allows to change current sharding on existing database.
Gives ceph-bluestore-tool 'reshard' command that allows to apply that to BlueStore.