CASSANDRA-21632: Read the BTI partition index preload in chunks - #5089
Open
aviau wants to merge 2 commits into
Open
CASSANDRA-21632: Read the BTI partition index preload in chunks#5089aviau wants to merge 2 commits into
aviau wants to merge 2 commits into
Conversation
aviau
force-pushed
the
bti-preload-chunked
branch
3 times, most recently
from
September 1, 2026 13:55
22b9616 to
f9968f7
Compare
Opening a BTI SSTable whose bloom filter is uninformative warms the whole Partitions.db, one byte per page through a reader that follows disk_access_mode. That costs one pread or one readahead-bounded page fault per page: on a 309 GiB index, 81.1M reads at 50 MiB/s regardless of access mode. It also warms far more than the available page cache can hold. Add cassandra.bti.partition_index_preload_size to bound how much is warmed. The tail is warmed because the trie is written bottom-up, so the upper levels traversed by every lookup are at the end of the file, while the bulk at the front is leaf pages that a lookup reads one of. The property takes a human-readable size, e.g. 512MiB. 0B skips warming entirely, which was not previously possible: preload is enabled whenever the bloom filter is uninformative, so the only way to avoid it was to lower bloom_filter_fp_chance below 1.0. A negative value warms the whole index and is the default, so behaviour is unchanged unless the property is set. Assisted-by: Claude Code:claude-opus-5 patch by Alexandre Viau; reviewed by TBD for CASSANDRA-21632
aviau
force-pushed
the
bti-preload-chunked
branch
from
September 1, 2026 20:53
f9968f7 to
389b5ae
Compare
driftx
requested changes
Sep 1, 2026
|
|
||
| // ChannelProxy.read may return a short read; advance by what was actually read. | ||
| int read = fh.channel.read(buffer, pos); | ||
| if (read <= 0) |
Contributor
There was a problem hiding this comment.
I think we should treat read < 0 as an EOFException, as FileDataInput.readByte() would have previously.
Touching one byte per page costs a pread or a page fault per page: 81.1M reads on a 309 GiB index. Read it in 1 MiB positioned reads on the channel instead, which are unaffected by disk_access_mode; page cache is shared between the buffered and mapped views of a file, so a mapped lookup still finds the data resident. The trade-offs: the whole range is copied into userspace rather than one byte per page, and under a mapped mode the first lookup takes a minor fault the walk had already resolved. Under disk_access_mode: standard the warm no longer populates the chunk cache, nor evicts it; lookups fill it on first touch instead. Assisted-by: Claude Code:claude-opus-5 patch by Alexandre Viau; reviewed by TBD for CASSANDRA-21632
aviau
force-pushed
the
bti-preload-chunked
branch
from
September 1, 2026 21:53
389b5ae to
47bb558
Compare
This was referenced Sep 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Meta
cassandra-6.0: CASSANDRA-21632: Bound and chunk the BTI partition index preload (for 6.0) #5091.trunk: CASSANDRA-21632: Bound and chunk the BTI partition index preload (for trunk) #5092.Description
Bounding the preload makes startup survivable, but touching one byte per page is still a slow way to warm a range: one pread or one page fault per page, 81.1M reads on my 309 GiB index. Reading in 1 MiB positioned reads on the channel gets the same pages resident at device bandwidth instead. Positioned reads are unaffected by disk_access_mode, and page cache is shared between the buffered and mapped views of a file, so a mapped lookup still finds the data resident.
This matters most when the warm is large, which is to say at the default (whole index) or at a large bound. With a small bound the walk is already cheap and this buys little.
Trade-offs:
Assisted-by: Claude Code:claude-opus-5
patch by Alexandre Viau; reviewed by TBD for CASSANDRA-21632