-
Notifications
You must be signed in to change notification settings - Fork 510
Closed
Description
Currently both iterators and compactions unconditionally send MERGE
keys' values through a roundtrip of the user-configured ValueMerger
, even when there is a single MERGE
key (so the value should be unchanged). This results in unnecessary allocations, especially in the context of CockroachDB and its allocation-heavy ValueMerger
implementation.
There are two potential resolutions:
- We could explicitly defer beginning the merging of values until we observe two internal keys with the same user key.
- We could adapt the compaction iterator to transform
MERGE
keys intoSET
s if we know a key is the oldest key at that user key (i.e.,allowZeroSeqNum=true
and the key is in the final snapshot stripe).
The code changes for approach (2) are a lot simpler, and changing the key to a SET has other advantages (eg, iteration can stop as soon as it hits the SET, avoiding an additional Next of the internal iterator).
Jira issue: PEBBLE-1139