There is no efficient way to merge two HLL Unions together.
HLL Union has the method update but that only takes HllSketch
To get a HllSketch from a HLL Union, we need to call HLL Union getResult which then calls gadget.copyAs(tgtHllType);
However, this does a deep copy, slowing down the merge.
Basically,
when this and other are both HLL Union:
→ other.union.getResult() // line 85: triggers gadget.copyAs() — FULL COPY of 65K registers
→ union.update(copiedSketch) // line 111: iterates 65K registers again to merge
Two passes over 65K registers + one allocation, when only one pass is needed.
Am I missing something? Thanks!
There is no efficient way to merge two HLL Unions together.
HLL Union has the method update but that only takes
HllSketchTo get a
HllSketchfrom a HLL Union, we need to call HLL UniongetResultwhich then callsgadget.copyAs(tgtHllType);However, this does a deep copy, slowing down the merge.
Basically,
when this and other are both HLL Union:
→ other.union.getResult() // line 85: triggers gadget.copyAs() — FULL COPY of 65K registers
→ union.update(copiedSketch) // line 111: iterates 65K registers again to merge
Two passes over 65K registers + one allocation, when only one pass is needed.
Am I missing something? Thanks!