Refactor StateStore for better readability and Giga support#2984
Refactor StateStore for better readability and Giga support#2984
Conversation
|
The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #2984 +/- ##
==========================================
+ Coverage 58.13% 58.34% +0.20%
==========================================
Files 2113 2121 +8
Lines 174071 174931 +860
==========================================
+ Hits 101204 102062 +858
- Misses 63812 63818 +6
+ Partials 9055 9051 -4
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| for _, db := range s.subDBs { | ||
| if err := db.SetLatestVersion(version); err != nil { | ||
| return err | ||
| } | ||
| } |
Check warning
Code scanning / CodeQL
Iteration over map Warning
| for _, db := range s.subDBs { | ||
| if v := db.GetEarliestVersion(); minVersion < 0 || v < minVersion { | ||
| minVersion = v | ||
| } | ||
| if err := db.ApplyBatch(pairs, version); err != nil { | ||
| return fmt.Errorf("failed to apply batch for %s: %w", StoreTypeName(storeType), err) | ||
| } |
Check warning
Code scanning / CodeQL
Iteration over map Warning
| for storeType, pairs := range grouped { | ||
| return s.applyToSubDB(storeType, version, pairs, async) | ||
| } |
Check warning
Code scanning / CodeQL
Iteration over map Warning
| go func(st EVMStoreType, p []*iavl.KVPair) { | ||
| defer wg.Done() | ||
| if err := db.ApplyBatch(pairs, version); err != nil { | ||
| if err := s.applyToSubDB(st, version, p, async); err != nil { | ||
| errCh <- err | ||
| } | ||
| }(db, pairs) | ||
| }(storeType, pairs) |
Check notice
Code scanning / CodeQL
Spawning a Go routine Note
* main: Harden `TestStateLock_NoPOL` against proposal/timeout race (#2980)
| "evmVersion", evmVersion, | ||
| "changelogPath", changelogPath, | ||
| ) | ||
| if err := ReplayWAL(logger, changelogPath, evmVersion, -1, func(entry proto.ChangelogEntry) error { |
There was a problem hiding this comment.
Curious, we do 2 separate replayWAls one for evmstore and cosmosStore. Wondering if we could just do 1 and do both inside each?
Not blocking, just curious
There was a problem hiding this comment.
It is definitely doable, I feel separating them is cleaner in the code and easier to remove later on, but the side effect is that it would require double read the WAL for some entries. Given that SS shouldn't fall behind too much (limited by buffer size), I think this should be fine to start with?
There was a problem hiding this comment.
Yea definitely fine to start with, could we just add a TODO to reconsider in future @yzang2019 ?
| ) | ||
|
|
||
| // Start migration if --migrate flag is set | ||
| if cast.ToBool(appOpts.Get("migrate-iavl")) { |
There was a problem hiding this comment.
it looks --migrate-iavl still changes runtime behavior?
There was a problem hiding this comment.
Yeah pretty sure no one is using that, it's for archive migration
This PR mainly add RocksDB support for SS and refactor for better readability and code structure: - Add New MvccDB interface for DB engine to unify pebbledb mvcc and rocksdb mvcc - StateStore interface unchanged and now extends MvccDB - Add a new CosmosStateStore: Thin wrapper around a single DB that implements StateStore. - EVMStateStore refactored: Sub-DBs typed as types.MvccDB (was types.StateStore) - CompositeStateStore simplified (ss/composite/store.go): Both cosmosStore and evmStore fields are now types.StateStore - not found errors moved to common/errors - RawImport and migrationKey interface removed - SS Migrator removed - db init static function removed, replaced by resolveBackend function - Fixed recoverFromWAL logic New structure: <img width="2816" height="1536" alt="image" src="https://github.com/user-attachments/assets/ec5ac9d1-b56f-4ea3-94fb-d7ad7a0f0afd" />
3 participants
Describe your changes and provide context
This PR mainly add RocksDB support for SS and refactor for better readability and code structure:
New structure:
Testing performed to validate your change