[hail] Write memory management layer in scala #7033
Conversation
|
Still waiting on getting benchmarks working. |
tpoterba
force-pushed
the
region-java-2
branch
from
c942706 to
ee2307a
Compare
|
Can we remove the corresponding Region C++ code, too? |
|
yes, will make that change afterwards. |
| p.check() | ||
| } | ||
|
|
||
| @Test def testRegionAllocationSimple() { |
There was a problem hiding this comment.
can you make sure that all of the (non-benchmark) tests in Region_test.cpp are accounted for here?
(also, maybe we should split these out into a separate RegionMemorySuite at some point)
There was a problem hiding this comment.
It might also be nice to add some checks on the total amount of memory allocated, since you're tracking that now.
| } | ||
|
|
||
| def isValid: Boolean = _isValid | ||
| final class Region(var blockSize: Region.Size, var pool: RegionPool, var memory: RegionMemory = null) extends AutoCloseable { |
There was a problem hiding this comment.
is this constructor intended to be used elsewhere? Might be worth making it private.
|
|
||
| def getReferenceCount: Long = referenceCount | ||
|
|
||
| def clear(): Unit = { |
There was a problem hiding this comment.
this should free large chunks.
There was a problem hiding this comment.
it feels like this and freeMemory want to do essentially the same thing, except that freeMemory also wants to release currentBlock and uninitialize the RegionMemory object by setting blockSize = -1. Might be worth breaking that part out into a separate function to make sure you're doing the same thing between the two calls.
|
|
||
| private def isFreed: Boolean = blockSize == -1 | ||
|
|
||
| def freeMemory(): Unit = { |
There was a problem hiding this comment.
it feels kind of weird to me that this is public, since it feels like it's mostly an internal function. Actually, I think RegionMemory itself could probably be package-private, which would let Region/RegionPool access all of its stuff without exposing the memory management elsewhere.
There was a problem hiding this comment.
yeah, that's what I want.
| o | ||
| } | ||
|
|
||
| def allocate(n: Long): Long = { |
There was a problem hiding this comment.
I kind of feel like we could remove this signature of allocate all the way up. (its only use is in appendByte which is only used in some basic region tests.) Reduces the duplicated logic in this PR, but you don't have to do it here.
|
|
||
| assert(pool.numRegions() == 1) | ||
| assert(pool.numFreeRegions() == 0) | ||
| assert(pool.numFreeBlocks() == 1) |
There was a problem hiding this comment.
I'm not sure why we'd expect to see a free block appearing here---if we're allocating size-1 bytes into an empty region, we should see it allocating into an existing block in the first case, and grabbing a large chunk in the second. When it gets cleared, we should keep the current block and there shouldn't be any used blocks to return to the pool; the large chunk should get freed independently.
tpoterba
force-pushed
the
region-java-2
branch
from
61b6df8 to
f83b8ac
Compare
4 participants
No description provided.