Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Already on GitHub? Sign in to your account

Mv bloom size limit #81

Merged
merged 8 commits into from Jun 17, 2013

Conversation

Projects
None yet
2 participants

MatthewVon added some commits Jun 5, 2013

MatthewVon Restrict total number of keys per sst/table file to reduce meta data. 26f81e3
MatthewVon remove mythical <3 and create new constant for levels in overlapped m…
…ode. Lots of changes to version_set.cc to make 2i not hit leveldb stall. Not completely happy with tuning of version_set.cc ... but it does survive tests now (at least on FusionIO).
35cee65
MatthewVon Create low threshold for compaction without penalty (throttle). Tune …
…special case of first sorted level.
4789b43
MatthewVon Remove dead code. Reduce penalty on first sorted layer to 3x from 6x. eef19a4
MatthewVon remove dead debug code e83aabd
MatthewVon notch the penalty (write throttle) factors down based on tests e102f10

@ghost ghost assigned matthewvon Jun 12, 2013

Matthew V close the leaking hatch ... crc32 test is NOT always active during co…
…mpaction ... now it is. This is to reduce disk corruption creating weird, random segfaults (and causing tech support calls).
bf620bd

@gburd gburd and 1 other commented on an outdated diff Jun 17, 2013

db/version_set.cc
for (outer=0; outer<files.size()-1 && !overlap_found; ++outer)
{
FileMetaData* outer_meta = files_[level][outer];
- const Slice outer_start = outer_meta->smallest.user_key();
+ //const Slice outer_start = outer_meta->smallest.user_key();
@gburd

gburd Jun 17, 2013

Any reason not to just delete this code rather than leave it commented out? If there's a reason to keep it around, maybe the comment should make note of that.

@matthewvon

matthewvon Jun 17, 2013

Contributor

Laziness. Will remove before push to master.

@gburd gburd and 1 other commented on an outdated diff Jun 17, 2013

db/version_set.cc
const Slice outer_limit = outer_meta->largest.user_key();
for (inner=outer+1; inner<files.size() && !overlap_found; ++inner)
{
FileMetaData* inner_meta = files_[level][inner];
const Slice inner_start = inner_meta->smallest.user_key();
- const Slice inner_limit = inner_meta->largest.user_key();
+ //const Slice inner_limit = inner_meta->largest.user_key();
@gburd

gburd Jun 17, 2013

Same comment here as for line 552, why keep this around?

@matthewvon

matthewvon Jun 17, 2013

Contributor

Laziness. Will remove before push to master.

gburd commented Jun 17, 2013

After reviewing and testing the code in the small (on my laptop) and then discussing with MvM these new heuristics for compaction and which levels may contain overlapping sst files I feel that these changes will improve things as anticipated and I don't see any obvious drawbacks. +1 to merge

@matthewvon matthewvon pushed a commit that referenced this pull request Jun 17, 2013

Matthew Von-Maszewski Merge pull request #81 from basho/mv-bloom-size-limit
Mv bloom size limit
be0d467

@matthewvon matthewvon merged commit be0d467 into master Jun 17, 2013

@matthewvon matthewvon deleted the mv-bloom-size-limit branch May 5, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment