Implement replay

[kvinwang]

Overview

This PR adds a new program replay comes with the following features:

• Fetch chain blocks and Replay the tokenomic computuing Just in-place without pRuntime needed.
• Write the workers ming finance events to a PostgresQL or TimescaleDB database.
• Provide a HTTP api to query worker's instant mining state.

Usage

$ ./target/release/replay -h
replay 0.1.0

USAGE:
    replay [OPTIONS]

FLAGS:
    -h, --help       Prints help information
    -V, --version    Prints version information

OPTIONS:
        --bind-addr <bind-addr>                    Bind address for local HTTP server. [default: 127.0.0.1:8080]
        --node-uri <node-uri>                      Substrate rpc websocket endpoint. [default: ws://localhost:9944]
        --persist-events-to <persist-events-to>    The PostgresQL database to store the events. [default: ]
        --start-at <start-at>                      The block number to start to replay at. [default: 413895]

Database

select block,time,event,v,payout from worker_finance_events where pubkey = '\xa436ab8f34f73f45b019248bb39b981cd118134afb897fa3c6a4437525ebeb1b' ORDER BY sequence;

 block  |          time           |        event        |        v         |    payout
--------+-------------------------+---------------------+------------------+---------------
 421104 | 2021-09-18 19:01:18.568 | mining_start        | 22642.4444767019 |             0
 421159 | 2021-09-18 19:13:48.402 | heartbeat_challenge | 22643.2788294261 |             0
 421163 | 2021-09-18 19:14:54.803 | heartbeat           | 22642.4444767019 |  0.8950341155
 421187 | 2021-09-18 19:20:54.347 | heartbeat_challenge | 22642.7976991938 |             0
 421192 | 2021-09-18 19:22:18.69  | heartbeat           | 22642.4444767019 |  0.4300107155
...
 437708 | 2021-09-21 21:04:18.601 | heartbeat_challenge | 22585.3801870510 |             0
 437718 | 2021-09-21 21:06:42.359 | heartbeat           | 22584.0893745792 |  1.4225321574
 437796 | 2021-09-21 21:24:18.3   | heartbeat_challenge | 22585.2887122098 |             0
 437807 | 2021-09-21 21:26:48.481 | enter_unresponsive  | 22585.4600513300 |             0
 439374 | 2021-09-22 03:31:30.336 | heartbeat           | 22467.7960355975 |             0
 439374 | 2021-09-22 03:31:30.336 | exit_unresponsive   | 22467.7960355975 |             0
 439574 | 2021-09-22 04:19:06.268 | heartbeat_challenge | 22467.9521989589 |             0

State API

curl localhost:9988/worker-state/a436ab8f34f73f45b019248bb39b981cd118134afb897fa3c6a4437525ebeb1b | jq
{
  "benchmarking": false,
  "current_block": 466583,
  "last_heartbeat_at_block": 466491,
  "last_heartbeat_for_block": 466485,
  "mining": true,
  "p_init": "2694",
  "p_instant": "2882.06203191890875802024",
  "unresponsive": false,
  "v": "22492.7475177576747557379",
  "v_init": "22642.44447670198867727826",
  "waiting_heartbeats": []
}

Attention

⚠️⚠️⚠️Note this PR changes some execution order in the GK, which should be carefully reviewed to see whether they're breaking changes. See the first commit for detail.
The origin code handle Mining events and Random number events in a block in the order of them generation. In this PR, the order changed to GK handle Random number in front and them handle all mining events.
Because the RandomNumber handling does not emit any mq message and it's state is isolated to mining state, I think this is not a breaking change.

Anyway, we should better carefully check the mq messages emitted from pRuntime and compare them to the previous versions to ensure it not get changed, before we release it.

[h4x3rotab]

Looks like we need to prepare a table first. Do you have the statement of the table scheme?

[kvinwang]

Looks like we need to prepare a table first. Do you have the statement of the table scheme?

https://github.com/Phala-Network/phala-blockchain/blob/796d00d5f508891676bae1fe03c90124576c9d22/standalone/replay/create_tables.sql