Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RocksDB Storage service #584

Merged
merged 5 commits into from
Feb 29, 2024
Merged

RocksDB Storage service #584

merged 5 commits into from
Feb 29, 2024

Conversation

al8n
Copy link
Contributor

@al8n al8n commented Feb 26, 2024

Implement RocksDB storage service, not use TransactionDB as it does not support open in read-only mode.

@al8n al8n added the enhancement New feature or request label Feb 26, 2024
@al8n al8n self-assigned this Feb 26, 2024
@al8n al8n mentioned this pull request Feb 26, 2024

/// Rocks transaction type
// Do not use `TransactionDB` here, because rocksdb's `TransactionDB` does not support open by read-only mode.
// Thus, we cannot open the same db in two or more processes.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, we cannot have a process writing and a process reading?

Copy link
Contributor Author

@al8n al8n Feb 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can because we use DB here, not the TransactionDB. rocksdb crate has two kinds of db, DB and TransactionDB, TransactionDB does not allow one read process while one write process. However, DB supports.

Copy link
Collaborator

@danielSanchezQ danielSanchezQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good. Can we add a test where we open the same DB from 2 processes, one opens and writes the other reads?

@al8n
Copy link
Contributor Author

al8n commented Feb 28, 2024

Looking good. Can we add a test where we open the same DB from 2 processes, one opens and writes the other reads?

I added a bin, so we can manually test it by using two processes, and there is also a unit test with one write thread, and multiple threads read.

Copy link
Collaborator

@danielSanchezQ danielSanchezQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good. Fix CI and ahead with it!! Thanks!

Copy link
Contributor

@youngjoon-lee youngjoon-lee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

async fn test_store_load_remove(
) -> Result<(), <RocksBackend<NoStorageSerde> as StorageBackend>::Error> {
let temp_path = TempDir::new().unwrap();
let sled_settings = RocksBackendSettings {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let sled_settings = RocksBackendSettings {
let rocks_settings = RocksBackendSettings {

also, in other tests

@al8n al8n merged commit fde0d29 into master Feb 29, 2024
7 checks passed
@al8n al8n deleted the storage-service branch February 29, 2024 04:32
al8n added a commit that referenced this pull request Mar 4, 2024
* finish rocksdb storage service
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants