Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storage Optimization 1: Measuring Write Amplification #47

Closed
maxwong opened this issue Mar 25, 2019 · 1 comment
Closed

Storage Optimization 1: Measuring Write Amplification #47

maxwong opened this issue Mar 25, 2019 · 1 comment
Assignees
Labels
arch Architecture related enhancement New feature or request

Comments

@maxwong
Copy link

maxwong commented Mar 25, 2019

Is your feature request related to a problem? Please describe.
Referring to #45 , this thread processes the following question:
Measuring write amplification.

Describe the solution you'd like
Writes different size of data in different number of keys to measure the amplification scale.

Describe alternatives you've considered

Additional context
We are using lmdb as storage engine. Will measure different engines to collect data and to compare.

@maxwong maxwong self-assigned this Mar 25, 2019
@maxwong maxwong added enhancement New feature or request arch Architecture related labels Mar 25, 2019
@maxwong
Copy link
Author

maxwong commented Mar 25, 2019

Preliminary tests indicates that storage usage is quite stable. Write amplification is at 1x level.
LMDB is a Copy-On-Write, B+ tree based storage engine. So the write amplification is quite stable (write size + log of data key size).

The storage engine used is LMDB. Found 2 articles (an open debate?):
Paul Banks - LMDB: The Leveldb Killer?
Response by the LMDB developer

These give us a better understanding of LMDB design considerations.

This was referenced Mar 28, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
arch Architecture related enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants