You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Lucene's ByteBlockPool and IntBlockPool are sort of like a little malloc implementation, embedding many logical growing int[] into a series of block buffers. This is needed to compactly/efficiently store postings data, since there can be many unique terms, each with their own growing byte[] holding postings.
IntBlockPool also has these slices, however, nowhere in core do we use those. Rather, the allocation needs for int[] is simpler: just allocated a fixed length 1, 2 or 3 int[] per unique term. We can greatly simplify IntBlockPool.
However, MemoryIndex does use these slices from IntBlockPool. I think we should move the complex slice logic in IntBlockPool out to MemoryIndex, simplifying core?
* [WIP] Move IntBlockPool slices to MemoryIndex
* [WIP] Working TestMemoryIndex
* [WIP} Working TestSlicedIntBlockPool
* Working many allocations tests
* Add basic IntBlockPool test
* SlicedIntBlockPool inherits from IntBlockPool
* Tidy
* [WIP] Move IntBlockPool slices to MemoryIndex
* [WIP] Working TestMemoryIndex
* [WIP} Working TestSlicedIntBlockPool
* Working many allocations tests
* Add basic IntBlockPool test
* SlicedIntBlockPool inherits from IntBlockPool
* Tidy
Spinoff from exciting issue #10712.
Lucene's
ByteBlockPool
andIntBlockPool
are sort of like a little malloc implementation, embedding many logical growingint[]
into a series of block buffers. This is needed to compactly/efficiently store postings data, since there can be many unique terms, each with their own growingbyte[]
holding postings.IntBlockPool
also has these slices, however, nowhere incore
do we use those. Rather, the allocation needs forint[]
is simpler: just allocated a fixed length 1, 2 or 3int[]
per unique term. We can greatly simplifyIntBlockPool
.However,
MemoryIndex
does use these slices fromIntBlockPool
. I think we should move the complex slice logic inIntBlockPool
out toMemoryIndex
, simplifying core?Migrated from LUCENE-10211 by Michael McCandless (@mikemccand)
The text was updated successfully, but these errors were encountered: