Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Relax VersionStorageInfo::GetOverlappingInputs check #4050

Closed
wants to merge 3 commits into from
Closed

Relax VersionStorageInfo::GetOverlappingInputs check #4050

wants to merge 3 commits into from

Commits on Jul 13, 2018

  1. Removed the unused TableCache::NewRangeTombstoneIterator

    @petermattis
    petermattis committed Jul 13, 2018
    Configuration menu
    Copy the full SHA
    008ff5b View commit details
    Browse the repository at this point in the history

  2. Truncate range tombstones from sstables 

    When adding range tombstones to a RangeDelAggregator, truncate the
tombstones to the sstable boundaries. This is done as an alternative to
truncating the range tombstones within the sstables themselves as it
properly handles existing data where range tombstones are not truncated
in sstables. Truncating range tombstones to sstable boundaries avoids
having two process all of the sstables with overlapping tombstones as a
unit.

See facebook#4050
    @petermattis
    petermattis committed Jul 13, 2018
    Configuration menu
    Copy the full SHA
    506a8d7 View commit details
    Browse the repository at this point in the history

  3. Relax VersionStorageInfo::GetOverlappingInputs check 

    Do not consider the range tombstone sentinel key as causing 2 adjacent
sstables in a level to overlap. When a range tombstone's end key is the
largest key in an sstable, the sstable's end key is set to a "sentinel"
value that is the smallest key in the next sstable with a sequence
number of kMaxSequenceNumber. This "sentinel" is guaranteed to not
overlap in internal-key space with the next sstable. Unfortunately,
GetOverlappingFiles uses user-keys to determine overlap and was thus
considering 2 adjacent sstables in a level to overlap if they were
separated by this sentinel key. This in turn would cause compactions to
be larger than necessary.

This previous behavior of GetOverlappingInputs was necessary due to the
following scenario:

  * Write a delete range [a, d).
  * After compaction, this deleted range may be added to multiple sst
    files: a.sst, b.sst, c.sst, but the boundaries of these sst files
    are [a, b), [b, c), [c, d).
  * When a.sst and b.sst reach the bottommost level, the delete range of
    the sst files will be dropped.
  * Write a new key in the range [a, c).
  * When the newly written key [a, c) reaches the bottommost level, its
    sequence number will be set to zero.
  * When the front c.sst compacts with the key in the range [a, c) the
    sequence number of that key is zero, and the key will be incorrectly
    dropped.

That scenario no longer occurs because we are truncating range deletion
tombstones to sstable boundaries when adding them to RangeDelAggregator.
    @petermattis
    petermattis committed Jul 13, 2018
    Configuration menu
    Copy the full SHA
    7363c90 View commit details
    Browse the repository at this point in the history