Originally posted by @lhotari in #26117 (comment)
Please notice that there's a need for a follow up PR to remove the RoaringBitSet (backed by RoaringBitmap) usage in org.apache.bookkeeper.mledger.impl.RangeSetWrapper.
There shouldn't be a need to choose between OpenLongPairRangeSet and LongPairRangeSet.DefaultRangeSet implementations in the Pulsar code base.
There's no need for a generic implementation of RangeSetWrapper in the Pulsar code base. All instances in production code are RangeSetWrapper<Position>. Therefore the implementation can be simplified significantly by replacing the generics solution in RangeSetWrapper with a new specific abstraction to handle Position and entryId/ledgerId with TreeMap & LongBitmap. Since there could be a need for locking of all methods already in this new abstraction, it's worth considering an LongBitmap implementation that doesn't add additional locking. Alternatively the new abstraction to replace RangeSetWrapper could directly wrap RoaringBitmap instead of using LongBitmap at all. That might be simpler after all.
Addressing this results in a larger refactoring.
Future work that depends on this change:
One of the goals of this refactoring is to support adding a more space-efficient individual deleted messages serialization for persisting managed cursor state. There's a related issue #25985. In addition to that, the key idea of PIP-81: Split the individual acknowledgments into multiple entries could be implemented to remove the upper limit for persisting managed cursor state. One of benefits of the solution is that the cursor state is split into multiple segments. When individual acks are persisted, there's a need to only persist the segments that have been modified since the last time the segments were persisted. The implementation plan and design of PIP-81 would need to be revisited in a new PIP so that it serves the purpose for the state of the current Pulsar code base.
Originally posted by @lhotari in #26117 (comment)
Addressing this results in a larger refactoring.
Future work that depends on this change:
One of the goals of this refactoring is to support adding a more space-efficient individual deleted messages serialization for persisting managed cursor state. There's a related issue #25985. In addition to that, the key idea of PIP-81: Split the individual acknowledgments into multiple entries could be implemented to remove the upper limit for persisting managed cursor state. One of benefits of the solution is that the cursor state is split into multiple segments. When individual acks are persisted, there's a need to only persist the segments that have been modified since the last time the segments were persisted. The implementation plan and design of PIP-81 would need to be revisited in a new PIP so that it serves the purpose for the state of the current Pulsar code base.