[v26.1.x] ct/l1: integrate metastore with cloud cache#30201
Merged
andrwng merged 11 commits intoredpanda-data:v26.1.xfrom Apr 16, 2026
Merged
[v26.1.x] ct/l1: integrate metastore with cloud cache#30201andrwng merged 11 commits intoredpanda-data:v26.1.xfrom
andrwng merged 11 commits intoredpanda-data:v26.1.xfrom
Conversation
Plumb an optional lowres_clock deadline through the reserve_space path so blocking waits respect it instead of waiting indefinitely. Upcoming work to integrate the metastore with the cloud cache will use this to bound reservation waits during staging file SST writes. (cherry picked from commit 56ab6c3)
Lets a single guard accumulate reservations from multiple chunks. Upcoming work will add a staging_file handle that reserves cache space incrementally as data is appended, consolidating each chunk into one guard so commits have exactly one reservation to finalize. (cherry picked from commit 9ad51e9)
Open files with O_EXCL instead of truncate in disk_persistence and cloud_persistence. Reject duplicate file handles in memory_persistence. The LSM engine guarantees unique file handles, so duplicates indicate a bug and should fail loudly. This is useful because I intend on adding another persistence type with this behavior, and it will be easier for it and the other persistence impls to have this guarantee, rather than having the overwrite behavior (especially since the LSM should guarantee unique IDs). (cherry picked from commit 630d2b8)
.part files represent in-flight writes whose space is already accounted for via reservations. Deleting them during trim_exhaustive would cause subsequent reads to fail, only to retry the download and recreate the file, so it doesn't really help much. Upcoming work will add a staging_file handle that creates new .part files for writes, making this race more likely. The original behavior was added (redpanda-data#11860) more as a pre-caution against hypothetical runtime bugs, rather than a functional fix of behavior. As is, .part files are cleaned up at cache startup anyway. (cherry picked from commit 41a3853)
Adds a handle bundling everything needed to write a file into the cache. Callers append data (cache space is reserved automatically in chunks), then commit or close depending on whether the appends were successful. This will be used to write SST staging files in an upcoming LSM persistence implementation that uses the cloud cache. The idea will be that SSTs will append to these staging files (rather than in the L1 staging directory) and then after uploading, commit them into the cloud cache so subsequent reads can use the local file. (cherry picked from commit 6f3a535)
Factor reusable code out of cloud_persistence.cc into cloud_data_persistence_base so the cache-backed implementation can reuse it. At a high level, this is code that interacts with the cloud directly, with the idea that we'll use this in a persistence implementation that is similar to the cloud persistence but uses cloud cache instead of the staging directory for local files. No behavior change. (cherry picked from commit bb20d8f)
Cache-backed data_persistence. Reads check the cache first and download on miss. Writes stage through the cache and upload to cloud, so subsequent reads hit locally. (cherry picked from commit c117cba)
replicated_db::open() takes cloud_io::cache* instead of a staging directory path. Thread the cache pointer through domain_supervisor and db_domain_manager in place of the staging directory. (cherry picked from commit 7dda78c)
The LSM previously staged SST files in the l1_staging directory via cloud_data_persistence. Now that it uses the cloud cache, those files are orphaned. Widen the startup cleanup to delete all regular files in the staging directory, not just .tmp files. (cherry picked from commit 98dc328)
Switch the read replica database_refresher to cache-backed LSM persistence. The staging directory is still passed through for l1::file_io which needs it for L1 object staging. (cherry picked from commit 5d92c77)
The cloud_persistence code is now superceded with cloud_cache_persistence. Updates the remaining callers (just tests) to use the cloud_cache_persistence instead of cloud_persistence, and moves the helpers and shared code into cloud_cache_persistence. (cherry picked from commit f4781ff)
andrwng
approved these changes
Apr 16, 2026
Collaborator
Author
CI test resultstest results on build#83275 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport of PR #30045