PebbleDB #230
Closed
PebbleDB #230
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
@@ Coverage Diff @@
## master #230 +/- ##
==========================================
- Coverage 68.54% 68.48% -0.06%
==========================================
Files 27 27
Lines 2130 2126 -4
==========================================
- Hits 1460 1456 -4
Misses 595 595
Partials 75 75
|
8 tasks
faddat
requested review from
alexanderbez,
cmwaters and
creachadair
as code owners
faddat
requested review from
tac0turtle,
tychoish and
williambanfield
as code owners
|
Remaining things to do here include adding pebbledb to the iavl benchmarks |
|
this contains a bug affecting goleveldb that we added while messing about. I'm going to close it and submit a pr that only touches pebble stuff. |
4 tasks
github-merge-queue bot
pushed a commit
to cometbft/cometbft
that referenced
this pull request
Jan 29, 2024
This PR integrates pebbledb, a key-value store by Cockroachdb based on Cockroach Labs findings about cgo: * https://www.cockroachlabs.com/blog/the-cost-and-complexity-of-cgo/ Unlike any other kv store that cometbft currently supports: * pebble is actively maintained unlike: * goleveldb * boltdb * badgerdb * pebble does not require the use of CGO like: * cleveldb * rocksdb In benchmarks: * pebble performs better than goleveldb, cleveldb and rocksdb * pebble performs consistently At Notional Ventures, Pte, we've used and submitted pebble for two years for: * archive nodes that would be crushed if they used goleveldb * high performance RPC infrastructure Here are the hitstorical pull requests to merge pebble: * tendermint/tm-db#230 * tendermint/tm-db#231 * tendermint/tm-db#281 * tendermint/tm-db#282 * tendermint/tm-db#283 * tendermint/tm-db#284 * tendermint/tm-db#304 * tendermint/tm-db#321 Pebble snapshots for cosmos blockchains are available here: * https://snapshot.notional.ventures/ --- #### PR checklist - [ ] Tests written/updated - [x] Changelog entry added in `.changelog` (we use [unclog](https://github.com/informalsystems/unclog) to manage our changelog) - [x] Updated relevant documentation (`docs/` or `spec/`) and code comments - [x] Title follows the [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec --------- Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
mergify bot
pushed a commit
to cometbft/cometbft
that referenced
this pull request
Jan 29, 2024
This PR integrates pebbledb, a key-value store by Cockroachdb based on Cockroach Labs findings about cgo: * https://www.cockroachlabs.com/blog/the-cost-and-complexity-of-cgo/ Unlike any other kv store that cometbft currently supports: * pebble is actively maintained unlike: * goleveldb * boltdb * badgerdb * pebble does not require the use of CGO like: * cleveldb * rocksdb In benchmarks: * pebble performs better than goleveldb, cleveldb and rocksdb * pebble performs consistently At Notional Ventures, Pte, we've used and submitted pebble for two years for: * archive nodes that would be crushed if they used goleveldb * high performance RPC infrastructure Here are the hitstorical pull requests to merge pebble: * tendermint/tm-db#230 * tendermint/tm-db#231 * tendermint/tm-db#281 * tendermint/tm-db#282 * tendermint/tm-db#283 * tendermint/tm-db#284 * tendermint/tm-db#304 * tendermint/tm-db#321 Pebble snapshots for cosmos blockchains are available here: * https://snapshot.notional.ventures/ --- #### PR checklist - [ ] Tests written/updated - [x] Changelog entry added in `.changelog` (we use [unclog](https://github.com/informalsystems/unclog) to manage our changelog) - [x] Updated relevant documentation (`docs/` or `spec/`) and code comments - [x] Title follows the [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec --------- Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com> (cherry picked from commit 82fa3c0)
melekes
added a commit
to cometbft/cometbft
that referenced
this pull request
Jan 29, 2024
This PR integrates pebbledb, a key-value store by Cockroachdb based on Cockroach Labs findings about cgo: * https://www.cockroachlabs.com/blog/the-cost-and-complexity-of-cgo/ Unlike any other kv store that cometbft currently supports: * pebble is actively maintained unlike: * goleveldb * boltdb * badgerdb * pebble does not require the use of CGO like: * cleveldb * rocksdb In benchmarks: * pebble performs better than goleveldb, cleveldb and rocksdb * pebble performs consistently At Notional Ventures, Pte, we've used and submitted pebble for two years for: * archive nodes that would be crushed if they used goleveldb * high performance RPC infrastructure Here are the hitstorical pull requests to merge pebble: * tendermint/tm-db#230 * tendermint/tm-db#231 * tendermint/tm-db#281 * tendermint/tm-db#282 * tendermint/tm-db#283 * tendermint/tm-db#284 * tendermint/tm-db#304 * tendermint/tm-db#321 Pebble snapshots for cosmos blockchains are available here: * https://snapshot.notional.ventures/ --- #### PR checklist - [ ] Tests written/updated - [x] Changelog entry added in `.changelog` (we use [unclog](https://github.com/informalsystems/unclog) to manage our changelog) - [x] Updated relevant documentation (`docs/` or `spec/`) and code comments - [x] Title follows the [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec --------- Co-authored-by: Anton Kaliaev <anton.kalyaev@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I spoke with the Terra team, and they mentioned that because their CosmWasm usage is high, there were issues with cgo/FFI (Rocks will hit this, as will CosmWasm, and this resulted in high overhead) when using RockDB, making it a less than ideal solution for their needs. Since CosmWasm is being adopted pretty widely in Cosmos, we're experimenting with PebbleDB, which outperforms RocksDB in synthetic benchmark tests.
Currently, the goal of these changes is to be able to test them and see if they increase RPC capacity on relayers, so that the IBC relay network can be more reliable.