After running a few benchmarks, 3000 looks like a reasonable value to keep HLLs with a few thousand elements small while the CPU cost is still not huge. This covers all the cases where the dense representation would use N orders of magnitude more space, like in the case of many HLLs with carinality of a few tens or hundreds. It is not impossible that in the future this gets user configurable, however it is easy to pick an unreasoable value just looking at savings in the space dimension without checking what happens in the time dimension.
It is safer since it is able to have side effects.
Even if it is a debugging command, make sure that when it forces a change in encoding, the command is propagated.
If we converted to dense, a register must be updated in the dense representation.
Mostly by reordering opcodes check conditional by frequency of opcodes in larger sparse-encoded HLLs.
Bottleneck found profiling. Big run time improvement found when testing after the change.
As more values are added splitting ZERO or XZERO opcodes, try to merge adjacent VAL opcodes if they have the same value.
Now the macros will work with arguments such as "ptr+1".
Bulk length for registers was emitted too early, so if there was a bug the reply looked like a long array with just one element, blocking the client as result.
We want to promote if the total string size exceeds the resulting size after the upgrade.
The function checks if all the HLL_REGISTERS were processed during the convertion from sparse to dense encoding, returning REDIS_OK or REDIS_ERR to signal a corruption problem. A bug in PFDEBUG GETREG was fixed: when the object is converted to the dense representation we need to reassign the new pointer to the header structure pointer.
Provides a human readable description of the opcodes composing a run-length encoded HLL (sparse encoding). The command is only useful for debugging / development tasks.
PFDEBUG will be the interface to do debugging tasks with a key containing an HLL object.
The new API takes directly the object doing everything needed to turn it into a dense representation, including setting the new representation as object->ptr.
sdsIncrLen() must be called anyway even if we are replacing the last oppcode of the sparse representation.
Two vars initialized to wrong values in createHLLObject().
The function didn't considered the fact that each XZERO opcode is two bytes.
Code never tested, but the basic layout is shaped in this commit. Also missing: 1) Sparse -> Dense conversion function. 2) New HLL object creation using the sparse representation. 3) Implementation of PFMERGE for the sparse representation.