Skip to content

compact: reduce allocations for lone MERGE keys #5178

@jbowens

Description

@jbowens

Currently both iterators and compactions unconditionally send MERGE keys' values through a roundtrip of the user-configured ValueMerger, even when there is a single MERGE key (so the value should be unchanged). This results in unnecessary allocations, especially in the context of CockroachDB and its allocation-heavy ValueMerger implementation.

There are two potential resolutions:

  1. We could explicitly defer beginning the merging of values until we observe two internal keys with the same user key.
  2. We could adapt the compaction iterator to transform MERGE keys into SETs if we know a key is the oldest key at that user key (i.e., allowZeroSeqNum=true and the key is in the final snapshot stripe).

The code changes for approach (2) are a lot simpler, and changing the key to a SET has other advantages (eg, iteration can stop as soon as it hits the SET, avoiding an additional Next of the internal iterator).

Jira issue: PEBBLE-1139

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions