Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace Zombienet with Chopsticks #657

Open
NachoPal opened this issue Jan 30, 2024 · 6 comments
Open

Replace Zombienet with Chopsticks #657

NachoPal opened this issue Jan 30, 2024 · 6 comments

Comments

@NachoPal
Copy link

NachoPal commented Jan 30, 2024

I am trying to replace Zombient with Chopsticks as a local testnet, but unfortunately I am facing some issues/limitations that make it not possible yet.

There are two different ways of using Chopsticks to deploy a network:

1. Running chains from genesis files

2. Forking production state.

  • Problem: Runtime upgrades are potentially very slow, and predicting their completion duration is not possible.
  • Solutions:
    • Having a local node with minimal latency for the RPC storage requests
      • It mitigates the problem, but runtimes upgrades could still take too long
    • Override the involved migrated storage data with some dummy data
      • It would't really test the production storage migration, plus needs manual intervention for each release. I am looking for an automated process that always works.
    • Add in the configuration file a method for pre-fetching the migrated storage data
      • Needs manual intervention for each release. I am looking for an automated process that always works.
    • Bulk query the keys to speed up the migration process? Possible? improve keysPaged #641
    • Make Chopsticks to work also with a copy of the production DB in your local
    • Add some kind of event I can subscribe to know when the migration has ended to resume running my tests. Currently, it produces a new block when running a runtime upgrade, however, Chopsticks stops working until the migrations finish. It would be great if I can pause my tests and resume them when Chopsticks is operative again.
      • While it doesn't resolve the problem of slow migrations, at least I can automate the process of running the tests.
    • Block should not be produced until migration is done.
      • While it doesn't resolve the problem of slow migrations, at least I can automate the process of running the tests.
@xlc
Copy link
Member

xlc commented Jan 30, 2024

you could try use https://github.com/open-web3-stack/parachain-launch to generate genesis with hrmp channel opened

the block production shouldn’t be too slow with runtime upgrade. one reason could make it slow is migration modifying a lot of data

@ermalkaleci
Copy link
Collaborator

#641
this will make migration run faster if you have specified a cache db

@NachoPal
Copy link
Author

Sorry, I posted the issue by mistake before completion. Please check it again.

@NachoPal
Copy link
Author

NachoPal commented Jan 30, 2024

one reason could make it slow is migration modifying a lot of data

Yes, exactly, that is the problem. I am trying to automate the testing process, but I can not really continue the testing flow after a runtime upgrade.

Add some kind of event I can subscribe to know when the migration has ended to resume running my tests
This might do the trick for a simple solution

The problem is that the block is produced, even the runtime spec_version updated, but Chopsticks is unresponsive because it is still doing the runtime migration. The block shouldn't be produced until the migration is done. That would also solve the issue, because I can await in my tests.

@NachoPal
Copy link
Author

  • Block should not be produced until migration is done.
    - While it doesn't resolve the problem of slow migrations, at least I can automate the process of running the tests.

I think this is probably the easiest and optimal solution.

@ermalkaleci
Copy link
Collaborator

not sure what you mean. block is not produced before migration

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants