You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, any range tombstone iterator passed to RangeDelAggregator is retained for the aggregator's entire lifetime. For long range scans, this can have a significant memory cost. Ideally, we would free iterators when the top-level DBIter moves past its corresponding file's range (L1+ only), and re-create them if DBIter moves back to the file. This would manifest as a RemoveTombstones method that is called by LevelIterator.
The problem with this approach is that a LevelIterator advances files after the last point key in the file is read, even if there is a gap between this key and the start of the next file that is covered by a range tombstone. We do not want to preemptively destroy the iterator containing this range tombstone, as it might cover keys in lower levels.
One solution to this problem is to insert sentinel keys between files in a LevelIterator if a file boundary has type kTypeRangeDeletion. These keys can be skipped by DBIter, but since this key will be stored in the underlying heap of iterators, it will keep the tombstones in the file alive for as long as they need to be alive.
The text was updated successfully, but these errors were encountered:
Currently, any range tombstone iterator passed to
RangeDelAggregator
is retained for the aggregator's entire lifetime. For long range scans, this can have a significant memory cost. Ideally, we would free iterators when the top-levelDBIter
moves past its corresponding file's range (L1+ only), and re-create them ifDBIter
moves back to the file. This would manifest as aRemoveTombstones
method that is called byLevelIterator
.The problem with this approach is that a
LevelIterator
advances files after the last point key in the file is read, even if there is a gap between this key and the start of the next file that is covered by a range tombstone. We do not want to preemptively destroy the iterator containing this range tombstone, as it might cover keys in lower levels.One solution to this problem is to insert sentinel keys between files in a
LevelIterator
if a file boundary has typekTypeRangeDeletion
. These keys can be skipped byDBIter
, but since this key will be stored in the underlying heap of iterators, it will keep the tombstones in the file alive for as long as they need to be alive.The text was updated successfully, but these errors were encountered: