[nhcb branch] Use single bit to differentiate between optimized bounds and floats #13828
Conversation
Use one bit to decide what kind of data to read/write. This reduces storage need of floats from 72 bits to 65 bits and makes the integers store in 17 instead of 16. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
|
LGTM, and passes all the unit tests of |
bboreham
changed the title
|
What is the Doesn't this make a breaking change to the storage on disk? |
What it is: It's not a breaking change, doesn't affect histogram chunk with the current schema for exponential histograms. |
|
Generally, I would say it helps a bit if we store things byte-aligned. Of course, it depends on the exact location in the bit stream if things are byte aligned at the position where we store the NHCB bounds. If generally not byte-aligned stuff has been in the stream before anyway, maybe we could also use the varbit (not varint) encoding for integers and be even a bit more compact. |
|
OK, I checked. We are definitely not byte-aligned at the time So I would like to modify the algorithm once again:
|
Update the limit under which we optimize since the new function is more permissive. Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
|
Actually, looking at how varbit is implemented there, maybe this optimisation could work? zenador@1f97f35 And once the implementation is finalised, we'd need to update the doc comments. |
Yes, this should work. |
Signed-off-by: Jeanette Tan <jeanette.tan@grafana.com>
Picked @zenador optimization, nice find! |
Signed-off-by: George Krajcsovits <krajorama@users.noreply.github.com>
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
|
LGTM |
Use one bit to decide what kind of data to read/write. This reduces storage need of floats from 72 bits to 65 bits and makes the integers store in 17 instead of 16.