Skip to content

DbLedgerStorage -- Main implementation#855

Closed
merlimat wants to merge 2 commits into
apache:masterfrom
merlimat:db-ledger-storage-impl
Closed

DbLedgerStorage -- Main implementation#855
merlimat wants to merge 2 commits into
apache:masterfrom
merlimat:db-ledger-storage-impl

Conversation

@merlimat
Copy link
Copy Markdown
Contributor

LedgerStorage implementation that uses the EntryLogger mechanism but stores the indexes in a RocksDB database.

In addition, there are also a WriteCache and a ReadCache that are used to completely decouple the read & write paths.

Copy link
Copy Markdown
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great feature!
I left some minor comments/questions


cleanupExecutor.execute(() -> {
try {
if (log.isDebugEnabled()) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This operation is out of any lock, if it takes very long time would it be possible to have concurrent clean ups running? Is this a problem?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not a problem because the cleanupExecutor is single threaded, so there will just be 1 cleanup happening at a given time.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll add a comment to clarify that in the code

System.out.println("newEntry3: " + ByteBufUtil.hexDump(newEntry3));
assertEquals(newEntry3, res);

storage.shutdown();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about closing storage in teardown() ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, fixed.

conf.setLedgerDirNames(new String[] { tmpDir.toString() });
Bookie bookie = new Bookie(conf);

storage = (DbLedgerStorage) bookie.getLedgerStorage();
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is never closed?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added shutdown in teardown method.

@merlimat merlimat force-pushed the db-ledger-storage-impl branch from 9ef0ec4 to 4f446e3 Compare December 15, 2017 01:12
@sijie
Copy link
Copy Markdown
Member

sijie commented Dec 15, 2017

@eolivelli can you review this pull request again?

Copy link
Copy Markdown
Contributor

@eolivelli eolivelli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 looks good

@merlimat merlimat closed this in bba1c6f Dec 15, 2017
ArvinDevel pushed a commit to ArvinDevel/bookkeeper that referenced this pull request Dec 19, 2017
`LedgerStorage` implementation that uses the EntryLogger mechanism but stores the indexes in a RocksDB database.

In addition, there are also a WriteCache and a ReadCache that are used to completely decouple the read & write paths.

Author: Matteo Merli <mmerli@apache.org>

Reviewers: Enrico Olivelli <eolivelli@gmail.com>, Sijie Guo <sijie@apache.org>

This closes apache#855 from merlimat/db-ledger-storage-impl
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants