Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New MantaPay Storage Layout and RPC #628

Merged
merged 9 commits into from
Jun 28, 2022
Merged

Conversation

stechu
Copy link
Contributor

@stechu stechu commented Jun 23, 2022

Description

closes: #603 #566

This PR implements a more optimized MantaPay storage layout from the benchmark result (#616). It contains the following major changes:

  • changes the Identity hash function in MantaPay to TwoxConcat
  • optimize the ledger RPC implementation by set an optimal parameter and change the pulling algorithm to a greedy algorithm
  • update typescript end to end test
  • unit test for RPC

Benchmark data: https://docs.google.com/spreadsheets/d/10BA-gE-NPjoNeCpMB_FmpQ7KkZkjfJVJKaVWCbKDqJg/edit#gid=0

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

  • Linked to Github issue with discussion and accepted design OR have an explanation in the PR that describes this work.
  • Wrote unit tests.
  • Updated relevant documentation in the code.
  • Added one line describing your change in <branch>/CHANGELOG.md
  • Re-reviewed Files changed in the Github PR explorer.
  • If runtime changes, need to update the version numbers properly:
    • authoring_version: The version of the authorship interface. An authoring node will not attempt to author blocks unless this is equal to its native runtime.
    • spec_version: The version of the runtime specification. A full node will not attempt to use its native runtime in substitute for the on-chain Wasm runtime unless all of spec_name, spec_version, and authoring_version are the same between Wasm and native.
    • impl_version: The version of the implementation of the specification. Nodes are free to ignore this; it serves only as an indication that the code is different; as long as the other two versions are the same then while the actual code may be different, it is nonetheless required to do the same thing. Non-consensus-breaking optimizations are about the only changes that could be made which would result in only the impl_version changing.
    • transaction_version: The version of the extrinsics interface. This number must be updated in the following circumstances: extrinsic parameters (number, order, or types) have been changed; extrinsics or pallets have been removed; or the pallet order in the construct_runtime! macro or extrinsic order in a pallet has been changed. You can run the metadata_diff.yml workflow for help. If this number is updated, then the spec_version must also be updated
  • Verify benchmarks & weights have been updated for any modified runtime logics
  • If importing a new pallet, choose a proper module index for it, and allow it in BaseFilter. Ensure every extrinsic works from front-end. If there's corresponding tool, ensure both work for each other.
  • If needed, update our Javascript/Typescript APIs. These APIs are officially used by exchanges or community developers.
  • If modifying existing runtime storage items, make sure to implement storage migrations for the runtime and test them with try-runtime. This includes migrations inherited from upstream changes, and you can search the diffs for modifications of #[pallet::storage] items to check for any.

@github-actions
Copy link

Please signoff on all commits with your name, email and gpg key for the Developer's Certificate of Origin.
git config --global user.name your_name
git config --global user.email you_email
COMMAND : git commit -s -S -m your_commit_message
-s = Signed-off-by
-S = Verify commit using gpg key
If there is more than one commit in your pull request and your git client is modern enough (2.13+), rebase the required number of commits with --signoff:
git rebase --signoff HEAD~<number_of_commits>
Then, force push:
git push -f origin <your_branch>
For instructions on managing gpg signature verification please visit: https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits

@stechu stechu marked this pull request as ready for review June 23, 2022 05:49
@stechu stechu force-pushed the fix/rpc-performance-boost branch 4 times, most recently from c16b9d2 to e8c7c2b Compare June 23, 2022 05:53
stechu and others added 2 commits June 23, 2022 01:54
RPC improvement

Signed-off-by: Shumo Chu <shumo.chu@protonmail.com>
Signed-off-by: Shumo Chu <shumo.chu@protonmail.com>
Signed-off-by: Shumo Chu <shumo.chu@protonmail.com>
Kevingislason
Kevingislason previously approved these changes Jun 23, 2022
Copy link

@Kevingislason Kevingislason left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

tests/README.md Outdated Show resolved Hide resolved
tests/manta_pay_rpc.ts Outdated Show resolved Hide resolved
Copy link
Contributor

@Garandor Garandor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partial review of non-test code

pallets/manta-pay/src/lib.rs Outdated Show resolved Hide resolved
pallets/manta-pay/src/lib.rs Show resolved Hide resolved
pallets/manta-pay/src/lib.rs Show resolved Hide resolved
pallets/manta-pay/src/lib.rs Show resolved Hide resolved
pallets/manta-pay/src/lib.rs Outdated Show resolved Hide resolved
pallets/manta-pay/src/rpc.rs Outdated Show resolved Hide resolved
@Garandor
Copy link
Contributor

just noticed the benchmark results only have 2 datasets. Shouldnt it be 3?

  • identityidentity
  • twoxidentity
  • twoxtwox

Signed-off-by: Shumo Chu <shumo.chu@protonmail.com>
Signed-off-by: Shumo Chu <shumo.chu@protonmail.com>
Signed-off-by: Shumo Chu <shumo.chu@protonmail.com>
Signed-off-by: Shumo Chu <shumo.chu@protonmail.com>
This was linked to issues Jun 28, 2022
Signed-off-by: Shumo Chu <shumo.chu@protonmail.com>
Signed-off-by: Shumo Chu <shumo.chu@protonmail.com>
Copy link
Contributor

@Garandor Garandor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved after discussion

@stechu stechu merged commit e36da8b into manta Jun 28, 2022
@stechu stechu deleted the fix/rpc-performance-boost branch June 28, 2022 18:17
@Apokalip Apokalip added the L-changed Log: Issues and PRs related to changes label Jul 28, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
L-changed Log: Issues and PRs related to changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Typescript Unittest For Ledger RPC Add RPC End-to-End Tests
5 participants