CASSANDRA-21632: Support bounding the BTI partition index preload - #5086
Open
aviau wants to merge 1 commit into
Open
CASSANDRA-21632: Support bounding the BTI partition index preload#5086aviau wants to merge 1 commit into
aviau wants to merge 1 commit into
Conversation
aviau
force-pushed
the
bti-preload-bound
branch
4 times, most recently
from
September 1, 2026 11:47
2ba2faa to
3ccfb02
Compare
driftx
requested changes
Sep 1, 2026
aviau
force-pushed
the
bti-preload-bound
branch
2 times, most recently
from
September 1, 2026 13:50
a9d55dd to
c5f22d2
Compare
driftx
requested changes
Sep 1, 2026
Opening a BTI SSTable whose bloom filter is uninformative warms the whole Partitions.db, one byte per page through a reader that follows disk_access_mode. That costs one pread or one readahead-bounded page fault per page: on a 309 GiB index, 81.1M reads at 50 MiB/s regardless of access mode. It also warms far more than the available page cache can hold. Add cassandra.bti.partition_index_preload_size to bound how much is warmed. The tail is warmed because the trie is written bottom-up, so the upper levels traversed by every lookup are at the end of the file, while the bulk at the front is leaf pages that a lookup reads one of. The property takes a human-readable size, e.g. 512MiB. 0B skips warming entirely, which was not previously possible: preload is enabled whenever the bloom filter is uninformative, so the only way to avoid it was to lower bloom_filter_fp_chance below 1.0. A negative value warms the whole index and is the default, so behaviour is unchanged unless the property is set. Assisted-by: Claude Code:claude-opus-5 patch by Alexandre Viau; reviewed by TBD for CASSANDRA-21632
aviau
force-pushed
the
bti-preload-bound
branch
from
September 1, 2026 20:38
c5f22d2 to
c7f0023
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Startup takes 90 minutes on my 5TB nodes with high partitions count.
Opening a BTI SSTable whose bloom filter is uninformative (bloom_filter_fp_chance = 1) warms the whole Partitions.db, one byte per page through a reader that follows disk_access_mode. That costs one pread or one readahead-bounded page fault per page: on a 309 GiB index, 81.1M reads. 90 minutes of startup time on my machine. It also warms far more than the available page cache can hold.
Add cassandra.bti.partition_index_preload_size to bound how much is warmed. The tail is warmed because the trie is written bottom-up, so the upper levels traversed by every lookup are at the end of the file, while the bulk at the front is leaf pages that a lookup reads one of.
The property takes a human-readable size, e.g. 512MiB:
Assisted-by: Claude Code:claude-opus-5
patch by Alexandre Viau; reviewed by TBD for CASSANDRA-21632
Note that I kept the change minimal to increase the chances of it getting merged. However, I think we could consider using a saner default for
cassandra.bti.partition_index_preload_size. Feel free to set another default when merging if you want.I also opened a follow-up PR here #5089 which changes the warmup so that it uses chunked reading.