New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Switch to explicit atomic_batch() database handle #1304
Conversation
We should expect performance gains once we start using write batches, as well. |
eth/db/backends/level.py
Outdated
self._write_batch.delete(key) | ||
del self._track_diff[key] | ||
|
||
def shutdown(self) -> None: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe rename to decomission
@@ -37,6 +46,136 @@ def test_set_and_get(memory_db, level_db): | |||
assert level_db.get(b'1') == memory_db.get(b'1') | |||
|
|||
|
|||
def test_atomic_batch(level_db, atomic_db): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rather than duplicating these here, do you think it makes sense to make this test suite generic or parametrized so that we can confirm feature parity between our in-memory databases and the leveldb ones?
eth/db/atomic.py
Outdated
|
||
@classmethod | ||
@contextmanager | ||
def commit_unexceptional(cls, write_target_db): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This name is odd, but am I correct that it's an internal API. If so, then I don't really care that much other than maybe prefixing with an _
to make that clear.
What was wrong?
The old approach of batching all changes made while leveldb was locked was flawed in a multi-threading context: it permits unintended writes to be batched with the ones you want.
How was it fixed?
Return an explicit handle when making an atomic batch, so that all the batched changes you want to make are explicitly part of that atomic batch. eg~
Cute Animal Picture