Compaction can be triggered manually by calling the
DB::CompactFiles method. This is meant to be used by advanced users to implement custom compaction strategies, including but not limited to the following use cases -
- Optimize for read heavy workloads by compacting to lowest level after ingesting a large amount of data
- Force the data to go through the compaction filter in order to consolidate it
- Migrate to a new compaction configuration. For example, if changing number of levels,
CompactRangecan be called to compact to bottommost level and then move the files to a target level
The example code below shows how to use the APIs.
Options dbOptions; DB* db; Status s = DB::Open(dbOptions, "/tmp/rocksdb", &db); // Write some data ... Slice begin("key1"); Slice end("key100"); CompactRangeOptions options; s = db->CompactRange(options, &begin, &end);
CompactionOptions options; std::vector<std::string> input_file_names; int output_level; ... Status s = db->CompactFiles(options, input_file_names, output_level);
end arguments define the key range to be compacted. The behavior varies depending on the compaction style being used by the db. In case of universal and FIFO compaction styles, the
end arguments are ignored and all files are compacted. Also, files in each level are compacted and left in the same level. For leveled compaction style, all files containing keys in the given range are compacted to the last level containing files. If either
end are NULL, it is taken to mean the key before all keys in the db or the key after all keys respectively.
If more than one thread calls manual compaction, only one will actually schedule it while the other threads will simply wait for the scheduled manual compaction to complete. If
CompactRangeOptions::exclusive_manual_compaction is set to true, the call will disable scheduling of automatic compaction jobs and wait for existing automatic compaction jobs to finish.
CompactRangeOptions supports the following options -
CompactRangeOptions::exclusive_manual_compactionWhen set to true, no other compaction will run when this manual compaction is running. Default value is
CompactRangeOptions::target_levelTogether, these options control the level where the compacted files will be placed. If
target_levelis -1, the compacted files will be moved to the minimum level whose computed max_bytes is still large enough to hold the files. Intermediate levels must be empty. For example, if the files were initially compacted to L5 and L2 is the minimum level large enough to hold the files, they will be placed in L2 if L3 and L4 are empty or in L4 if L3 is non-empty. If
target_levelis positive, the compacted files will be placed in that level provided intermediate levels are empty. If any any of the intermediate levels are not empty, the compacted files will be left where they are.
CompactRangeOptions::target_path_idCompaction outputs will be placed in options.db_paths[target_path_id] directory.
CompactRangeOptions::bottommost_level_compactionWhen set to
BottommostLevelCompaction::kSkip, or when set to
BottommostLevelCompaction::kIfHaveCompactionFilterand a compaction filter is defined for the column family, the bottommost level files are not compacted.
This API compacts all the input files into a set of output files in the
output_level. The number of output files is determined by the size of the data and the setting of
CompactionOptions::output_file_size_limit. This API is not supported in ROCKSDB_LITE.