Replace Zombienet with Chopsticks

[NachoPal]

I am trying to replace Zombient with Chopsticks as a local testnet, but unfortunately I am facing some issues/limitations that make it not possible yet.

There are two different ways of using Chopsticks to deploy a network:

1. Running chains from genesis files

• Problem: HRMP not working for genesis mode after opening channels #634
• Solution: hrmp from unregistered parachains #635

2. Forking production state.

• Problem: Runtime upgrades are potentially very slow, and predicting their completion duration is not possible.
• Solutions:
  • Having a local node with minimal latency for the RPC storage requests
    • It mitigates the problem, but runtimes upgrades could still take too long
  • Override the involved migrated storage data with some dummy data
    • It would't really test the production storage migration, plus needs manual intervention for each release. I am looking for an automated process that always works.
  • Add in the configuration file a method for pre-fetching the migrated storage data
    • Needs manual intervention for each release. I am looking for an automated process that always works.
  • Bulk query the keys to speed up the migration process? Possible? improve keysPaged #641
  • Make Chopsticks to work also with a copy of the production DB in your local
  • Add some kind of event I can subscribe to know when the migration has ended to resume running my tests. Currently, it produces a new block when running a runtime upgrade, however, Chopsticks stops working until the migrations finish. It would be great if I can pause my tests and resume them when Chopsticks is operative again.
    • While it doesn't resolve the problem of slow migrations, at least I can automate the process of running the tests.
  • Block should not be produced until migration is done.
    • While it doesn't resolve the problem of slow migrations, at least I can automate the process of running the tests.

[xlc]

you could try use https://github.com/open-web3-stack/parachain-launch to generate genesis with hrmp channel opened

the block production shouldn’t be too slow with runtime upgrade. one reason could make it slow is migration modifying a lot of data

[ermalkaleci]

#641
this will make migration run faster if you have specified a cache db

[NachoPal]

Sorry, I posted the issue by mistake before completion. Please check it again.

[NachoPal]

one reason could make it slow is migration modifying a lot of data

Yes, exactly, that is the problem. I am trying to automate the testing process, but I can not really continue the testing flow after a runtime upgrade.

Add some kind of event I can subscribe to know when the migration has ended to resume running my tests
This might do the trick for a simple solution

The problem is that the block is produced, even the runtime spec_version updated, but Chopsticks is unresponsive because it is still doing the runtime migration. The block shouldn't be produced until the migration is done. That would also solve the issue, because I can await in my tests.

[NachoPal]

• Block should not be produced until migration is done.
  - While it doesn't resolve the problem of slow migrations, at least I can automate the process of running the tests.

I think this is probably the easiest and optimal solution.

[ermalkaleci]

not sure what you mean. block is not produced before migration