index: Use roaring bitmaps for content posting lists

[keegancsmith]

No description provided.

[keegancsmith]

@sourcegraph/core-services so this might not be better. I believe loading a roaring bitset requires scanning over the whole []buf representing it. This may actually be way more IO than the current delta encoding approach, which lazily delta decodes. So in the case of something with a lot of results, you will do way less IO. But in the case of something with few results, you may end up doing the same amount of IO.

So roaring bitsets may be inappropriate for posting lists to byte offsets. There is likely something better though. From a quick look at the roaring bitsets implementation, we can probably get away with something based on roaring bitsets which is optimized for streaming them, or we can look into some research.

[tsenart]

I believe loading a roaring bitset requires scanning over the whole []buf representing it.

From: https://github.com/RoaringBitmap/RoaringFormatSpec#general-layout

The layout is designed so that random access to the data is possible without materializing the bitmap in memory while being storage efficient.