Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(tests/scripts): create script to retrieve trie state via rpc #3714

Merged
merged 18 commits into from
Jan 29, 2024

Conversation

jimjbrettj
Copy link
Contributor

@jimjbrettj jimjbrettj commented Jan 18, 2024

Changes

  • Created a script to query a running node via rpc to retrieve the trie state at a given block and write it to a file
  • I also added the ability to compute and verify the state root from the retrieved state

Tests

Start up a gossamer node and execute the script against it (or any running gossamer node)

Issues

#3713

Primary Reviewer

@EclesioMeloJunior

Copy link

codecov bot commented Jan 18, 2024

Codecov Report

Attention: 55 lines in your changes are missing coverage. Please review.

Comparison is base (4566b14) 50.56% compared to head (a8f77d0) 50.45%.

Additional details and impacted files
@@               Coverage Diff               @@
##           development    #3714      +/-   ##
===============================================
- Coverage        50.56%   50.45%   -0.12%     
===============================================
  Files              230      231       +1     
  Lines            28654    28728      +74     
===============================================
+ Hits             14489    14494       +5     
- Misses           12649    12711      +62     
- Partials          1516     1523       +7     

@jimjbrettj jimjbrettj added T-enhancement this issue/pr covers improvement of existing functionality. S-tests issue related to adding new tests. S-sync-westend related to particular network syncing. T-feat this issue/pr is a new feature or functionality. and removed T-enhancement this issue/pr covers improvement of existing functionality. labels Jan 18, 2024
Copy link
Member

@EclesioMeloJunior EclesioMeloJunior left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice script! 🔥 🚀 🚀

Few comments and suggestions to improve the script versatility, also could you write down some test cases?

tests/scripts/trieStateScript.go Outdated Show resolved Hide resolved
tests/scripts/trieStateScript.go Outdated Show resolved Hide resolved
tests/scripts/trieStateScript.go Outdated Show resolved Hide resolved
tests/scripts/trieStateScript.go Outdated Show resolved Hide resolved
tests/scripts/trieStateScript.go Outdated Show resolved Hide resolved
tests/scripts/trieStateScript.go Outdated Show resolved Hide resolved
@dimartiro
Copy link
Contributor

What if we move it to a scripts folder (at root level) instead of tests/scripts
Also we can create a make script to run it

tests/scripts/trieStateScript.go Outdated Show resolved Hide resolved
@jimjbrettj
Copy link
Contributor Author

Responded to all feedback except @dimartiro given how simple of a script this is I did not see the benefit of using a makefile

Copy link
Contributor

@dimartiro dimartiro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think using trie.TrieLayout could make it more scalable and safety.

Also, I think the file name should be trie_state_script.go and trie_state_script_test.go

scripts/trieStateScript.go Outdated Show resolved Hide resolved
scripts/trieStateScript.go Outdated Show resolved Hide resolved
scripts/trieStateScript.go Outdated Show resolved Hide resolved
scripts/trieStateScript.go Outdated Show resolved Hide resolved
@jimjbrettj jimjbrettj merged commit 5ccea40 into development Jan 29, 2024
24 checks passed
@jimjbrettj jimjbrettj deleted the jimmy/trieStateScript branch January 29, 2024 18:36
github-actions bot pushed a commit that referenced this pull request Mar 1, 2024
# [0.9.0](v0.8.0...v0.9.0) (2024-3-1)

### Bug Fixes

* add a limit of number of bytes while scale decoding a slice ([#3733](#3733)) ([5edbf89](5edbf89))
* **docs:** Fixing link to polkadot runtime fundamentals to the right one ([#3763](#3763)) ([a785d32](a785d32))
* don't panic if we fail to convert hex to bytes ([#3734](#3734)) ([12234de](12234de))
* **dot/sync:** execute p2p handshake when there is no target ([#3695](#3695)) ([a9db0ec](a9db0ec))
* fix index out of range undeterministic error in rpc test ([#3718](#3718)) ([d099384](d099384))
* fix non deterministic  panic during TestStableNetworkRPC integration test ([#3756](#3756)) ([ee3d243](ee3d243))
* **lib/trie:** use `MustBeHashed` for V1 trie nodes with larger storage values ([#3739](#3739)) ([f5e48a9](f5e48a9))
* **mocks:** Set fixed version for uber mockgen in CI ([#3656](#3656)) ([ea9877e](ea9877e))
* **runtime/storage:** support nested storage transactions ([#3670](#3670)) ([3e99f6d](3e99f6d))
* segfault on node restart ([#3736](#3736)) ([d1ca7aa](d1ca7aa))
* **state-version:** should be uint8 instead of uint32 ([#3779](#3779)) ([c8fdb14](c8fdb14))
* update paseo chain spec ([#3770](#3770)) ([6a54f28](6a54f28))
* use last finalized block on startup ([#3737](#3737)) ([c262642](c262642))

### Features

* **config:** dynamically set version based on environment ([#3693](#3693)) ([5c534c9](5c534c9))
* **staging:** Expose RPC on Westend Staging Node ([#3687](#3687)) ([c374eaa](c374eaa))
* **tests/scripts:** create script to retrieve trie state via rpc ([#3714](#3714)) ([5ccea40](5ccea40))
Copy link

github-actions bot commented Mar 1, 2024

🎉 This PR is included in version 0.9.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

timwu20 pushed a commit that referenced this pull request Apr 19, 2024
# [0.9.0](v0.8.0...v0.9.0) (2024-3-1)

### Bug Fixes

* add a limit of number of bytes while scale decoding a slice ([#3733](#3733)) ([5edbf89](5edbf89))
* **docs:** Fixing link to polkadot runtime fundamentals to the right one ([#3763](#3763)) ([a785d32](a785d32))
* don't panic if we fail to convert hex to bytes ([#3734](#3734)) ([12234de](12234de))
* **dot/sync:** execute p2p handshake when there is no target ([#3695](#3695)) ([a9db0ec](a9db0ec))
* fix index out of range undeterministic error in rpc test ([#3718](#3718)) ([d099384](d099384))
* fix non deterministic  panic during TestStableNetworkRPC integration test ([#3756](#3756)) ([ee3d243](ee3d243))
* **lib/trie:** use `MustBeHashed` for V1 trie nodes with larger storage values ([#3739](#3739)) ([f5e48a9](f5e48a9))
* **mocks:** Set fixed version for uber mockgen in CI ([#3656](#3656)) ([ea9877e](ea9877e))
* **runtime/storage:** support nested storage transactions ([#3670](#3670)) ([3e99f6d](3e99f6d))
* segfault on node restart ([#3736](#3736)) ([d1ca7aa](d1ca7aa))
* **state-version:** should be uint8 instead of uint32 ([#3779](#3779)) ([c8fdb14](c8fdb14))
* update paseo chain spec ([#3770](#3770)) ([6a54f28](6a54f28))
* use last finalized block on startup ([#3737](#3737)) ([c262642](c262642))

### Features

* **config:** dynamically set version based on environment ([#3693](#3693)) ([5c534c9](5c534c9))
* **staging:** Expose RPC on Westend Staging Node ([#3687](#3687)) ([c374eaa](c374eaa))
* **tests/scripts:** create script to retrieve trie state via rpc ([#3714](#3714)) ([5ccea40](5ccea40))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-sync-westend related to particular network syncing. S-tests issue related to adding new tests. T-feat this issue/pr is a new feature or functionality.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants