import hardspoon account balances from standalone chain into genesis block of parachain

[ltfschoen]

Status update:

• need to resolve the Error 1 mentioned below.
  • See https://gist.github.com/ltfschoen/44db460c02b23aa6562e3567cfac4659
  • See https://substrate.stackexchange.com/questions/156/how-to-overwrite-endowed-account-balances-using-additional-accounts-and-balances
• need to replace the contents of genesis.json with the actual list of hardspooned accounts and balances that were obtained from the Subscan API, which has been uploaded here https://github.com/DataHighway-DHX/open-polkadot-js-library/blob/main/subscan/data/datahighway.api.subscan.io-api-scan-accounts-2022-02-03-12:09-993000000-genesis-fixture.json, so we just copy/paste across
• fix so DHX DAO unlocked reserves is 30 million DHX (as it was incorrect)

Error 1

i've created this PR #5 to combine both the chain specification's endowed accounts with the contents of a JSON file genesis.json that contains further accounts and balances that i want to be credited to them in the genesis block (to overwrite any values in the endowed accounts).

it compiles successfully, after changing to the compatible Rust version and compiling the code

./scripts/init.sh
cargo build --release

but when i then follow the polkadot-launch steps to copy the binary to the relevant polkadot-launch folder after setting up polkadot-launch to use DataHighway using this PR DataHighway-DHX/polkadot-launch#2 and then run polkadot-launch, the polkadot-launch logs give an error message when it generates the chain spec Error: 0: Other: Error parsing spec file: 0x prefix is missing at line 208 column 1756,

then copying the binary to the relevant folder for use with polkadot-launch :

cp ./target/release/datahighway-collator ~/code/DataHighway-DHX/polkadot-launch/bin/datahighway-collator

and then running the final step to run polkadot-launch with:

yarn start config.json

it outputs error:

🧹 Resolving parachain id...
2022-02-10 14:05:34 Building chain spec    

🧹 Starting with a fresh authority set...
  👤 Added Genesis Authority alice
  👤 Added Genesis Authority bob
  👤 Added Genesis Authority charlie
  👤 Added Genesis Authority dave

⚙ Updating Relay Chain Genesis Configuration
  ✓ Updated Genesis Configuration [ validation_upgrade_frequency: 10 ]
  ✓ Updated Genesis Configuration [ validation_upgrade_delay: 10 ]

⛓ Adding Genesis Parachains
  ✓ Added Genesis Parachain 2000
  ✓ Added Genesis Parachain 3000
⛓ Adding Genesis HRMP Channels
  ✓ Added HRMP channel 2000 -> 3000
Added Boot Nodes: /ip4/127.0.0.1/tcp/30336/p2p/12D3KooWF66thRTDXG2ksKm3nMCHeoMneuohs71Q4anPLVnBSsSp,/ip4/127.0.0.1/tcp/30337/p2p/12D3KooWAZ1VKfix84y8rrrLekd6xVksDgjiYooTNTBi32GSXCWN,/ip4/127.0.0.1/tcp/30338/p2p/12D3KooWEpmRWnuEga2oFPZ7P8BfcJBZj928XT46hhj6zyieA72R,/ip4/127.0.0.1/tcp/30339/p2p/12D3KooWFUiU4LrYD1MVwbvkE4wqvCTy2feU9LijMKPoEXBJhArG

2022-02-10 14:05:36 Building chain spec    
Error: 
   0: Other: Error parsing spec file: 0x prefix is missing at line 208 column 1756
...

which is because the rococo-local.json file that it generates isn't being generated correctly, and its including snippets of the actual code in front of the hex values of the "genesis_head" value and "validation_code" properties as shown below:

rococo-local.json

...
"paras": {
  "paras": [
    [
      2000,
      {
        "genesis_head": "account_public_key_json "a42b7518d62a942344fec55d414f1654bf3fd325dbfa32a3c30534d5976acb21"\naccount_balance_json 60000000000000000000000\naccount_ss58_address_json a42b7518d62a942344fec55d414f1654bf3fd325dbfa32a3c30534d5976acb21 (5FmxcuFw...)\nno match b/w endowed and json for "a42b7518d62a942344fec55d414f1654bf3fd325dbfa32a3c30534d5976acb21" so adding it to the list\n0x0000000000000000000000000000000000000000000000000000000000000000005c76496ba72...",
        "validation_code": "account_public_key_json "a42b7518d62a942344fec55d414f1654bf3fd325dbfa32a3c30534d5976acb21"\naccount_balance_json 60000000000000000000000\naccount_ss58_address_json a42b7518d62a942344fec55d414f1654bf3fd325dbfa32a3c30534d5976acb21 (5FmxcuFw...)\nno match b/w endowed and json for "a42b7518d62a942344fec55d414f1654bf3fd325dbfa32a3c30534d5976acb21" so adding it to the list\n0x52bc5376

i can't seem to debug in the chain_spec.rs file to figure out why it's not working (i.e. using println! or log::trace, etc), so i replicated the relevant parts of the code in a separate rust project here to see what was going on https://github.com/ltfschoen/RustTest/tree/master/projects/endow
but that appears to be combining and returning the accounts and balances correctly:

[
  (
    a42b7518d62a942344fec55d414f1654bf3fd325dbfa32a3c30534d5976acb21 (5FmxcuFw...),
    30000000000000000000000,
  ),
...
]

so i tried to isolate the issue...

if i only use the for loop https://github.com/DataHighway-DHX/DataHighway-Parachain/pull/5/files#diff-f2256017c4e606a30f53708578da46675512cc1300be6542ed5f77e58af0d6dcR961 and remove the line that calls get_allocation so it doesn't also add the accounts and balances in the .json file to the genesis, then it generates an error Error parsing spec file: 0x prefix is missing at line when i use polkadot-launch,

but if i don't use that for loop (so it doesn't credit the endowed accounts), and i uncomment the get_allocation function again (so it credits the accounts loaded from the genesis.json file), then it compiles successfully and polkadot-launch launches the relay chain and collator nodes successfully, but it doesn't credit the accounts loaded from the genesis.json file when i add them to the address book to check their balance with polkadot.js at https://polkadot.js.org/apps/?rpc=ws%3A%2F%2F127.0.0.1:9988#/addresses

• - problem:
  need to find out who is being endowed in the parachain chain_spec that's causing the total issuance to double

research:
the in-built treasury account from the standalone chain, which is also the in-built treasury account for the parachain
ss58 address: 4LTFqiD6H6g8a7ur9WH4RxhWx2givWfK7o5EDed3ai1nYTvk
public key: 6d6f646c70792f74727372790000000000000000000000000000000000000000
which already has a balance of:
30.182709334742898651223000 million DHX

in the genesis.json file that is a backup of the standalone chain balances we have:
total issuance from the standalone chain of:
3.1852973655698758055894e+25 (i.e. 31.8 million DHX)

but when i start the new parachain, and check the balances > totalIssuance, it is:
61,957,973,655,698,758,055,894,000 (i.e. 61 million DHX)

solution:
this was because the old temporary treasury account a42b7518d62a942344fec55d414f1654bf3fd325dbfa32a3c30534d5976acb21 was also being funded in chain_spec.rs, which it shouldn't have, so i have removed it.

we'll be using the balance in the genesis.json for the treasury, which will 4LTFqiD6H6g8a7ur9WH4RxhWx2givWfK7o5EDed3ai1nYTvk, which will overwrite the default value in the chain_spec.rs

• - problem: the sudo account from the standalone chain won't be the sudo account in the parachain, as we're using different keys for it. the old sudo standalone account is:
  ss58 address: 4NN9N4NCLWQWNsb2RRYE7CSPTN1tLa7Ez5eViJ4N1Q5wx9z3
  public key: c201d4551d04a99772d8efe196490a96b4ee5e608ac8e495be9505a99e723069
  it already has a balance of:
  9.519994858881105000 DHX

solution:
we could remove the balance from the genesis.json and update the js script so it is clear that it was removed, so it isn't given a balance in the parachain. but i think we should just leave it and a democracy proposal can be used to burn that amount in the future instead since it is a small amount

• the authorities from the standalone chain won't also be authorities on the parachain as they have different accounts and keys,
  but these were effectively run by the DataHighway (funded by MXC) to run validator nodes and to secure the standalone chain.
  so although those accounts were given a head start with ~10 DHX in the initial standalone genesis, i think the account balances that they have now should remain since they contributed to early development.
  their account balances have changed due to validator rewards, so the old authority account balances are:
  note that their gran, babe, imon and audi keys didn't have an account balance.

authority 1

stash
3c62839f3ce86df3c27c66016b092d15186b23e429db4ad8338501fc219bfa6c
~10 DHX

controller
42b728d9c752fe87c3e3db40d9a7d02f22b81bc1f0e49c59a5e128b861f87b08
~22,760 DHX

authority 2
stash
628580490f55f5340d8a5e5af85eb78fdc066f6ac0385ccb3d17dc9a8b3e651f
~10 DHX

controller
e627945747e5a66afd5ff3f383819dfb6dd9633f6c9d52b24b10307cc841d577
~23,672 DHX

authority 3
stash
44d784f6d346d337a98e273f683590c661ea20b83aa6c9ac93754e02cee4372e
~10 DHX

controller
32e28b6f06a728b13d782b007d57ab53d7fe3fa2d5def2c58585d27007d5ea05
~7,869 DHX

authority 4

stash
1e7c18d12311c46ca4fd9fba48f11ea81c7c87e992fc5b1abb4903219cc6b429
~22,704.8269 DHX

controller
3ee5ed5cdf314660b60c6acb87d51af5f10c5d8ea24dd762df4c5973a1683416
~10 DHX

authority 5

stash
b0e6b234c77cc3d9fde0f3a039779c8d6ae0a66cb95d764acc54667e74f4c800
~10,690 DHX

controller
2643aaa4cf95aa0f014c015722e31a356eb8bd17888f74bca1ca56c404445c39
~10 DHX

authority 6

stash
48e771e75097d353d06b5fd469edf8cb53d8c9b8eeeb1aa0f480ff7a42e27f28
~342 DHX

cont
3e028f22ca42f5feee5344c3319ba97d23019087a124b43a825cb7d35ed7d522
~10 DHX

authority 7

stash
1ae9a95688b6dcf6db83c61789e59352cdf363c5b13d39f37c27f7434439027e
~6,341 DHX

cont
76ea25fc43fbdd113efabf6a12ea1f67c2916f9dd15d7c08a020269aa28cd521
~10 DHX

authority 8

stash
ec9ee2c38014483a454cfe108cc062e0ff641fef7c5c10f0a0f797cfdb860708
~3,984 DHX

cont
ee323b965d01799f4af213347549ab5e8e533071df69c4d2ed122a354deb930c
~10 DHX

authority 9

stash
746be7c22192aa11d46d3b2bdad13cc77b1bab69ab9eb10799b629dd7ddfbf7e
~4,144 DHX

cont
cc517121c11d0135836a62816526bb40d9c9a0ae47f316c646c148a5cff0f200
~10 DHX

authority 10
stash
ecc71d63ef4b80d8feab189a764cd3e759f941062f9b07168bbfcb814da7bf33
~7,625 DHX

cont
3c71cfbd77668301af5aefe0c81e3b4ffc1c0f0b07c0d42044922237d574455f
~10 DHX

[ayushmishra2005]

@ltfschoen Can you perform local testing using https://docs.substrate.io/tutorials/v3/cumulus/start-relay/ and let me know if it is working fine

[ltfschoen]

@ltfschoen Can you perform local testing using https://docs.substrate.io/tutorials/v3/cumulus/start-relay/ and let me know if it is working fine

https://docs.substrate.io/tutorials/v3/cumulus/start-relay/

it works fine with local testing using "rococo-local" by following these steps https://gist.github.com/ltfschoen/2e7b3f5508094b4179908507a8dffe58