You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
It seems to me that it may be nice to chunk the structure into simpler blocks, made up of (Offset, Chunk Offset Dictionary, Bits), encoding those into the array, and then repeating. This has the benefit of being better for cache locality from Offset -> Chunk Offset Dictionary. It would also help with locality when digging into the actual bits, but I suspect it wouldn't be as big of a gain.
For reference, we currently encode the entire level 1 dictionary, then the entire level2 dictionary, and then all the bits.
Also, we have some space in the level 2 dictionary, since we are aligning on Int boundaries (2 bits per 96 bits). We could use this to encode some extra information to let us switch between implementations. For example, if that chunk has < X bits (2^16, 2^8, etc), we could save some space in the storage.
The text was updated successfully, but these errors were encountered:
It seems to me that it may be nice to chunk the structure into simpler blocks, made up of
(Offset, Chunk Offset Dictionary, Bits)
, encoding those into the array, and then repeating. This has the benefit of being better for cache locality fromOffset -> Chunk Offset Dictionary
. It would also help with locality when digging into the actual bits, but I suspect it wouldn't be as big of a gain.For reference, we currently encode the entire level 1 dictionary, then the entire level2 dictionary, and then all the bits.
Also, we have some space in the level 2 dictionary, since we are aligning on Int boundaries (2 bits per 96 bits). We could use this to encode some extra information to let us switch between implementations. For example, if that chunk has < X bits (2^16, 2^8, etc), we could save some space in the storage.
The text was updated successfully, but these errors were encountered: