CASSANDRA-21019 – 6.0 – Separate memtable memory usage tracking from limit checking - #5049
Closed
michaelsembwever wants to merge 4 commits into
Closed
CASSANDRA-21019 – 6.0 – Separate memtable memory usage tracking from limit checking#5049michaelsembwever wants to merge 4 commits into
michaelsembwever wants to merge 4 commits into
Conversation
Brings this branch in line with the review of the trunk PR, apache#5050, where the comments were left. Identify nested writes by write context, not by the updateIndexes flag. Index build, index cleanup during compaction and stale entry purge all write with updateIndexes == false, each on its own context, so each must still wait for pool room. CassandraWriteContext now marks the outermost memtable write, and ColumnFamilyStore.apply routes a write on an already-marked context to putNested. Replaces MemtableNestedPutSkipsRoomWaitTest, which mutated the shared MEMORY_POOL counters, with NestedMemtableWriteTest. Reverts the ColumnFamilyStore.apply and TableWriteHandler javadoc that described updateIndexes as the nesting signal. Reject memtable_offheap_space: 0 at startup when memtable_allocation_type is offheap_buffers or offheap_objects. awaitRoom skips a sub-pool whose limit is 0, which is needed for the heap-only allocation types, but which would leave off-heap memtable memory unlimited for these two. Such a node could not make progress on writes before this fix, so no working deployment has that config. Documented in cassandra.yaml and NEWS.txt, with tests in DatabaseDescriptorTest. Correct the comments the reviewer read as inaccurate: awaitRoom's zero-limit guard no longer claims that such a pool is never allocated from or never signalled, and MemtablePool.limit says the same; put() is gated per PartitionUpdate rather than per mutation, and its wait is unconditional, so a deletion or a write that loses on timestamp waits too; putNested's precondition is a barrier on Keyspace.writeOrder, which is shared by every table, so the flush that deadlocks need not be of the memtable being written. markExtraOnHeapUsed and markExtraOffHeapUsed no longer promise to block. Unlike trunk, this branch keeps the OpOrder.Group parameter on SubAllocator.allocate, SubAllocator.adjust, Memtable.markExtraOnHeapUsed and Memtable.markExtraOffHeapUsed. The group is unused now that these calls never wait, but removing it would change a published interface inside a release series. The javadoc says so at each one.
michaelsembwever
force-pushed
the
mck/21019/6.0
branch
from
August 28, 2026 22:35
74c31c9 to
c4d0f66
Compare
… in checkSpaceAndPut() Brings this branch level with mck/21019/trunk, which took two further comments on apache#5050 after the previous squash. Rename the API so that the method an implementer writes is the plain one. Memtable.put is now the ungated write, AbstractAllocatorMemtable.performPut is gone, and the entry point a write starting a mutation calls is Memtable.checkSpaceAndPut. ColumnFamilyStore.apply, AccordKeyspace, TrieMemtableShardLockDeadlockTest and the three memtable implementations follow. Behaviour is unchanged: the same call sites gate, and a nested write still reaches put() directly. Add the MemtableAllocatorAwaitRoomTest case for the other half of the invariant. Accounting an allocation must record and return while the pool is over its limit rather than park, which is the path markExtraOnHeapUsed takes. Trunk's remaining review commit also drops the now-unused OpOrder.Group from SubAllocator.allocate, SubAllocator.adjust, Memtable.markExtraOnHeapUsed and Memtable.markExtraOffHeapUsed. This branch does not, for the reason recorded in the previous squash: those are published methods inside a release series. Assisted-by: Claude Code:claude-opus-5
blambov
approved these changes
Sep 11, 2026
netudima
approved these changes
Sep 13, 2026
Member
Author
|
committed as 2152f78 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.