Mv level work2 #76

Merged
merged 2 commits into from Apr 5, 2013

Conversation

Projects
None yet
2 participants
Contributor

matthewvon commented Apr 5, 2013

This branch provides further tuning to the mv-level-work1 branch. The tuning is focused upon reducing when write throttle penalties apply and the performance between levels 2 and 3. The level 2 to 3 transition is when the code shifts from overlapped .sst files to sorted. The change slightly increases the write amplification (bad) but reduces latencies, both overall and worst case (good).

MatthewVon added some commits Apr 2, 2013

MatthewVon Heavy rework of penalty factors. These are close, but likely one more…
… adjustment. Mostly helps worst case latencies, not overall throughput.
1940c39
matthewvon tune level 3 to have size approximate to what level 2 will dump into …
…it. Do not start penalties on sorted levels until 2.5 times limit.
730beb8

matthewvon was assigned Apr 5, 2013

I've reviewed the code enough to understand what is going on and visually detect any fat fingering and such.

Besides the new tuning magic numbers, we have:

  • Penalties not triggering so soon, especially when just reaching # file compaction threshold
  • The previous changes to allow levels beyond 0 to have overlapping files was not been completely effective because the code in Compaction:ShouldStopBefore was triggering and smaller compaction files were generated, which ended up hurting performance when compaction through the first non-overlapping level occurred

👍 💃

@matthewvon matthewvon pushed a commit that referenced this pull request Apr 5, 2013

Matthew Von-Maszewski Merge pull request #76 from basho/mv-level-work2
Mv level work2
b4b2d27

@matthewvon matthewvon merged commit b4b2d27 into master Apr 5, 2013

engelsanchez was assigned Apr 8, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment