storage: allow compactor to delete sstable files in compacted ranges #24137
Conversation
This is a test. DO NOT MERGE.
|
This change is |
|
No warranties on this. I haven't even tried it out on a real example. It's just recycled code from the original PR, refurbished for the current state of the compactor. I'm unclear on the source of the slowdown, so it's not obvious to me that this will address the root problem. |
| // If the key spans don't overlap, then check whether they're | ||
| // "nearly" contiguous. | ||
| if aggr.EndKey.Compare(sc.StartKey) < 0 { | ||
| // First, if voided, the spans must be exactly contiguous or we | ||
| // are unable to aggregate. | ||
| if aggr.Voided { |
There was a problem hiding this comment.
I guess this is where we'll end up not being able to determine contiguous ranges for a single delete files in range for a much larger key span, unfortunately.
FYI, the original code in the PR to efficiently drop tables sent the full key span of the table being dropped as part of the suggested compaction so this gap could be bridged with some extra logic in the compactor.
There was a problem hiding this comment.
No, this wouldn't be enough. Sending the span along only proves that the key space was supposed to be empty at some past instant in time. Replicas could have reappeared into that empty keyspace and you wouldn't be allowed to delete them. I think what we'd need really is a tighter coupling with the store. Whenever the compactor finds a gap it wants to bridge, it asks the store how far it can go (and takes the min with how far it wants to go). The store adds placeholders for any handed-out keyspan until the compactor acks them back.
This is a test. DO NOT MERGE.