Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.Sign up
Added a AppendingDeltaPackedLongBuffer-based storage format to single value field data #5706
The AppendingDeltaPackedLongBuffer uses delta compression in paged fashion. For data which is roughly monotonic this results in a reduced memory signature.
By default we use the storage format expected to use the least memory. You can force a choice using a new field data setting
Running some benchmarks on simulated time based data shows 25-30% reduction in memory usage with a very small performance overhead (current implementations uses PACKED as a memory format with 0.5 acceptable overhead ratio):