Add CpcUnion.update(MemorySegment)

[davecromberge]

What

CpcUnion only accepts a CpcSketch, so merging a stored sketch means deserializing it first:

union.update(CpcSketch.heapify(MemorySegment.ofArray(bytes)));

That builds an entire sketch — sliding window, pair table, HIP registers — which the union then walks once and discards. This adds an overload that skips it:

union.update(MemorySegment.ofArray(bytes));

Where the union already holds a bit matrix, the image's coupons are decoded straight into it. Sparse, Hybrid and Pinned images decode directly. Sliding partly inverts its logic — a coupon can be signalled by the absence of a pair — so it still goes through a sketch. Either way the resulting union is byte-identical.

Use case

Unioning columns of serialized sketches, which is what query engines and offline rollup jobs mostly do. In Pinot this path is taken by distinctCountRawCpcSketch, by segment rollup, and by star-tree index construction; all three heapify every input today.

How much it helps depends on the flavor of the stored sketches, which follows their cardinality — see below.

Measured

New characterization profile CpcUnionDeserializeSpeedProfile — 32 stored sketches merged per trial at lgK=12 — median of 4 interleaved runs of each entry point against the same jar. It sweeps 29 unique counts from 256 to 4.2M, grouped here by the flavor the stored images take:

flavor of the stored images	change
Sparse	−28.3%, geometric mean over 12 unique counts
Hybrid	−32.7%, geometric mean over 5
Pinned	−3.5%, geometric mean over 8
Sliding (falls back to a sketch)	+0.7%, geometric mean over 4

The saving tracks how many of an image's coupons live in the surprises table rather than the sliding window, because both paths decode the window identically:

• Sparse and Hybrid hold every coupon as a pair, so decoding straight into the bit matrix removes a pair-table build and walk proportional to the whole sketch.
• Pinned keeps nearly everything in the window — at lgK=12 and 262144 uniques the images average 30 pairs against 6085 coupons — so there is little table work left to remove.
• Sliding takes the fallback, and its +0.7% is this harness's noise floor, since both configurations run identical code there.

The gain is largest for columns of many small sketches, and falls away for columns of few large
ones, where the union pays for a window decode either way.