storage: add traits and a memory implementation #86
Conversation
|
do we need to present local/remote/external category concepts in code structure? if so do we need to move |
No, it's not necessary. We can provide some built-in implementations in As for cloud platforms, maybe we can consider adding a |
ok, I'd like to implement s3 in |
|
@zojw I refactored the file layout a little bit. Specifically, I move storage traits to an |
huachaohuang
changed the title
|
|
||
| /// An interface to upload an object. | ||
| #[async_trait] | ||
| pub trait StorageObjectUploader { |
There was a problem hiding this comment.
maybe this trait is more suitable laid in storage_object.rs~~~?
ditto MemObjectUploader
There was a problem hiding this comment.
But actually storage_object.rs doesn't need this, it is the return type of a StorageBucket method, so I think it should be considered as a part of StorageBucket.
| pub trait ObjectHandle {} | ||
| pub use async_trait::async_trait; | ||
|
|
||
| // TODO: use std::stream::Stream instead |
There was a problem hiding this comment.
May we create an issue for it and note how to achieve it? TODO in comments can be always an issue in the project which is more actionable and comment-able.
There was a problem hiding this comment.
Make senses, here is the issue #91
| Box::new(stream::iter(object_names)) | ||
| } | ||
|
|
||
| async fn upload_object(&self, name: &str) -> Box<dyn StorageObjectUploader> { |
There was a problem hiding this comment.
maybe we need to detect the state of the bucket that owned this object?
e.g. two people A and B
---- A: get bucket "bucket-test" that exists and hold MemBucket to next usage
---- B: empty and delete "bucket-test"
---- A: try to upload object into "bucket-test" <=== should return an error, because the object can not download?
There was a problem hiding this comment.
I think it depends. If we mimic file system behaviors on Linux, it is OK to use an opened but deleted directory/file. Once the directory is closed, all data is gone. But I think we should follow the practice of different object storages here.
Refactor storage interfaces as described in #80.
This PR also provides a memory storage implementation.
More tests may be added later.
Ref: #68