Add PaginatedKVStore traits upstreamed from ldk-server #4347
Conversation
|
👋 Thanks for assigning @tnull as a reviewer! |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #4347 +/- ##
==========================================
- Coverage 86.08% 86.04% -0.05%
==========================================
Files 156 156
Lines 102416 102624 +208
Branches 102416 102624 +208
==========================================
+ Hits 88165 88298 +133
- Misses 11759 11830 +71
- Partials 2492 2496 +4
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
lightning/src/util/persist.rs
Outdated
|
|
||
| /// A token that can be used to retrieve the next set of keys. | ||
| /// The `String` is the last key from the current page and the `i64` is | ||
| /// the associated `time` used for ordering. |
There was a problem hiding this comment.
I don't think most DBs are going to support this? "sort by the last-changed as of time X" isn't possible without storing what the last-updated time was prior to X when something is updated after X.
There was a problem hiding this comment.
ldk-server does it based on creation_at not updated_at, will update the docs to make that more clear
There was a problem hiding this comment.
Hmm, do all of our backend storage things have a created-at time? I guess if you want it time-ordered you have to do it that way but it adds quite a requirement :/
Allows for a paginated KV store for more efficient listing of keys so you don't need to fetch all at once. Uses monotonic counter or timestamp to track the order of keys and allow for pagination. The traits are largely just copy-pasted from ldk-server. This also adds a PaginatedKVStoreSyncAdapter/PaginatedKVStoreAdapter so you can use a paginated kv store in contexts that expect a regular key value store. Adds some basic tests that were generated using claude code.
benthecarman
force-pushed
the
paginated-kv-store
branch
from
5160a11 to
3fb869f
Compare
| pub keys: Vec<String>, | ||
|
|
||
| /// A token that can be used to retrieve the next set of keys. | ||
| /// The `String` is the last key from the current page and the `i64` is |
There was a problem hiding this comment.
I'm not sure I understand why this field is here, honestly. This doc indicates that the first element is just repeating an entry from keys (why can't I just call keys.last()?) and the second element doesn't seem super useful either (is there some DB where we need it for some reason?)
| /// when dealing with a large number of keys that cannot be efficiently retrieved all at once. | ||
| /// | ||
| /// For an asynchronous version of this trait, see [`PaginatedKVStore`]. | ||
| pub trait PaginatedKVStoreSync: Send + Sync { |
There was a problem hiding this comment.
Why doesn't this just extend KVStoreSync? Seems weird to duplicate the interface.
3 participants
Allows for a paginated KV store for more efficient listing of keys so you don't need to fetch all at once. Uses monotonic counter or timestamp to track the order of keys and allow for pagination. The traits are largely just copy-pasted from ldk-server.
This also adds a PaginatedKVStoreSyncAdapter/PaginatedKVStoreAdapter so you can use a paginated kv store in contexts that expect a regular key value store.
Adds some basic tests that were generated using claude code.