-
Notifications
You must be signed in to change notification settings - Fork 94
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
linera-views
: add local variants of store traits
#1843
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -295,8 +295,8 @@ pub trait KeyValueIterable<Error> { | |
} | ||
|
||
/// Low-level, asynchronous read key-value operations. Useful for storage APIs not based on views. | ||
#[async_trait] | ||
pub trait ReadableKeyValueStore<E> { | ||
#[trait_variant::make(ReadableKeyValueStore: Send)] | ||
pub trait LocalReadableKeyValueStore<E> { | ||
/// The maximal size of keys that can be stored. | ||
const MAX_KEY_SIZE: usize; | ||
|
||
|
@@ -324,33 +324,44 @@ pub trait ReadableKeyValueStore<E> { | |
/// Finds the `(key,value)` pairs matching the prefix. The prefix is not included in the returned keys. | ||
async fn find_key_values_by_prefix(&self, key_prefix: &[u8]) -> Result<Self::KeyValues, E>; | ||
|
||
// We can't use `async fn` here in the below implementations due to | ||
// https://github.com/rust-lang/impl-trait-utils/issues/17, but once that bug is fixed | ||
// we can revert them to `async fn` syntax, which is neater. | ||
|
||
/// Reads a single `key` and deserializes the result if present. | ||
async fn read_value<V: DeserializeOwned>(&self, key: &[u8]) -> Result<Option<V>, E> | ||
fn read_value<V: DeserializeOwned>( | ||
&self, | ||
key: &[u8], | ||
) -> impl Future<Output = Result<Option<V>, E>> | ||
where | ||
Self: Sync, | ||
E: From<bcs::Error>, | ||
{ | ||
from_bytes_opt(&self.read_value_bytes(key).await?) | ||
async { from_bytes_opt(&self.read_value_bytes(key).await?) } | ||
} | ||
|
||
/// Reads multiple `keys` and deserializes the results if present. | ||
async fn read_multi_values<V: DeserializeOwned + Send>( | ||
fn read_multi_values<V: DeserializeOwned + Send>( | ||
&self, | ||
keys: Vec<Vec<u8>>, | ||
) -> Result<Vec<Option<V>>, E> | ||
) -> impl Future<Output = Result<Vec<Option<V>>, E>> | ||
where | ||
Self: Sync, | ||
E: From<bcs::Error>, | ||
{ | ||
let mut values = Vec::with_capacity(keys.len()); | ||
for entry in self.read_multi_values_bytes(keys).await? { | ||
values.push(from_bytes_opt(&entry)?); | ||
async { | ||
let mut values = Vec::with_capacity(keys.len()); | ||
for entry in self.read_multi_values_bytes(keys).await? { | ||
values.push(from_bytes_opt(&entry)?); | ||
} | ||
Ok(values) | ||
} | ||
Ok(values) | ||
} | ||
} | ||
|
||
/// Low-level, asynchronous write key-value operations. Useful for storage APIs not based on views. | ||
#[async_trait] | ||
pub trait WritableKeyValueStore<E> { | ||
#[trait_variant::make(WritableKeyValueStore: Send)] | ||
pub trait LocalWritableKeyValueStore<E> { | ||
/// The maximal size of values that can be stored. | ||
const MAX_VALUE_SIZE: usize; | ||
|
||
|
@@ -363,10 +374,10 @@ pub trait WritableKeyValueStore<E> { | |
} | ||
|
||
/// Low-level trait for the administration of stores and their namespaces. | ||
#[async_trait] | ||
pub trait AdminKeyValueStore: Sized { | ||
#[trait_variant::make(AdminKeyValueStore: Send)] | ||
pub trait LocalAdminKeyValueStore: Sized { | ||
/// The error type returned by the store's methods. | ||
type Error: Send; | ||
type Error; | ||
|
||
/// The configuration needed to interact with a new store. | ||
type Config: Send + Sync; | ||
|
@@ -378,11 +389,14 @@ pub trait AdminKeyValueStore: Sized { | |
async fn list_all(config: &Self::Config) -> Result<Vec<String>, Self::Error>; | ||
|
||
/// Deletes all the existing namespaces. | ||
async fn delete_all(config: &Self::Config) -> Result<(), Self::Error> { | ||
for namespace in Self::list_all(config).await? { | ||
Self::delete(config, &namespace).await?; | ||
fn delete_all(config: &Self::Config) -> impl Future<Output = Result<(), Self::Error>> { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is that really absolutely needed? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. See the comment here — it's because There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. (I would also prefer to keep it as There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Pertinent |
||
async { | ||
let namespaces = Self::list_all(config).await?; | ||
for namespace in namespaces { | ||
Self::delete(config, &namespace).await?; | ||
} | ||
Ok(()) | ||
} | ||
Ok(()) | ||
} | ||
|
||
/// Tests if a given namespace exists. | ||
|
@@ -395,26 +409,30 @@ pub trait AdminKeyValueStore: Sized { | |
async fn delete(config: &Self::Config, namespace: &str) -> Result<(), Self::Error>; | ||
|
||
/// Initializes a storage if missing and provides it. | ||
async fn maybe_create_and_connect( | ||
fn maybe_create_and_connect( | ||
config: &Self::Config, | ||
namespace: &str, | ||
) -> Result<Self, Self::Error> { | ||
if !Self::exists(config, namespace).await? { | ||
Self::create(config, namespace).await?; | ||
) -> impl Future<Output = Result<Self, Self::Error>> { | ||
async { | ||
if !Self::exists(config, namespace).await? { | ||
Self::create(config, namespace).await?; | ||
} | ||
Self::connect(config, namespace).await | ||
} | ||
Self::connect(config, namespace).await | ||
} | ||
|
||
/// Creates a new storage. Overwrites it if this namespace already exists. | ||
async fn recreate_and_connect( | ||
fn recreate_and_connect( | ||
config: &Self::Config, | ||
namespace: &str, | ||
) -> Result<Self, Self::Error> { | ||
if Self::exists(config, namespace).await? { | ||
Self::delete(config, namespace).await?; | ||
) -> impl Future<Output = Result<Self, Self::Error>> { | ||
async { | ||
if Self::exists(config, namespace).await? { | ||
Self::delete(config, namespace).await?; | ||
} | ||
Self::create(config, namespace).await?; | ||
Self::connect(config, namespace).await | ||
} | ||
Self::create(config, namespace).await?; | ||
Self::connect(config, namespace).await | ||
} | ||
} | ||
|
||
|
@@ -426,6 +444,18 @@ pub trait KeyValueStore: | |
type Error: Debug; | ||
} | ||
|
||
/// Low-level, asynchronous write and read key-value operations, without a `Send` bound. Useful for storage APIs not based on views. | ||
pub trait LocalKeyValueStore: | ||
LocalReadableKeyValueStore<Self::Error> + LocalWritableKeyValueStore<Self::Error> | ||
{ | ||
/// The error type. | ||
type Error: Debug; | ||
} | ||
|
||
impl<S: KeyValueStore> LocalKeyValueStore for S { | ||
type Error = <Self as KeyValueStore>::Error; | ||
} | ||
|
||
#[doc(hidden)] | ||
/// Iterates keys by reference in a vector of keys. | ||
/// Inspired by https://depth-first.com/articles/2020/06/22/returning-rust-iterators/ | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We use
fn() -> impl Future
instead ofasync fn()
here because default implementations ofasync fn
confusetrait_variant
, but it seems fine with default implementations of-> impl Trait
fn
s.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it make sense to add a comment in the code?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it should be okay since if someone tries to change this it will fail to compile.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(and this applies to several places, so I'd have to copy-paste the comment several times in order to make sure the reader caught it)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think Mathieu Baudet meant adding a TODO(#XXX) that corresponds to considering the resolution of that bug. There are several pending issues here like the confusion of the trait and having a mention could help in dealing with that. The idea is that if something needs you 10 minutes to figure out then a comment is worthwhile.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't add an issue because we're currently blocked on the upstream bug so there's not really much for us TODO — but I've added a comment above the default implementations with a link to the bug to save future readers the time of figuring it out :)