Revise `near_indexer::StreamerMessage` struct #1

khorolets · 2022-01-31T08:19:55Z

We assume StreamerMessage contains duplicates we want to avoid. It should revise the struct to find out what can be optimized there or dropped from.

The text was updated successfully, but these errors were encountered:

khorolets · 2022-01-31T09:25:12Z

https://hackmd.io/@uYOqIgyuTWytaOXsS1rgxA/HkAeeQBCF

@telezhnaya @frol please, have a look at the StreamerMessage revision and let me know if you have concerns or objections to my conclusion about it.

khorolets · 2022-02-08T18:12:50Z

This PR on nearcore side near/nearcore#6255 should resolve our basic concerns about StreamerMessage struct. But we'll resolve them later

khorolets · 2022-02-10T18:08:28Z

Current StreamerMessage (serde_json) early testnet block so missing a lot of regular data

Object({
    "block": Object({
        "author": String(
            "node1",
        ),
        "chunks": Array([
            Object({
                "balance_burnt": String(
                    "0",
                ),
                "chunk_hash": String(
                    "Ayiv3wZNHGhbsyf8ReKpeJYDgugWTiwgWGZDU4iN3aZW",
                ),
                "encoded_length": Number(
                    8,
                ),
                "encoded_merkle_root": String(
                    "5TxYudsfZd2FZoMyJEZAP19ASov2ZD43N8ZWv8mKzWgx",
                ),
                "gas_limit": Number(
                    1000000000000000,
                ),
                "gas_used": Number(
                    0,
                ),
                "height_created": Number(
                    42376924,
                ),
                "height_included": Number(
                    42376925,
                ),
                "outcome_root": String(
                    "11111111111111111111111111111111",
                ),
                "outgoing_receipts_root": String(
                    "H4Rd6SGeEBTbxkitsCdzfu9xL9HtZ2eHoPCQXUeZ6bW4",
                ),
                "prev_block_hash": String(
                    "fQURSjwQKZn8F98ayQjpndh85msJBu12FBkUY1gc5WA",
                ),
                "prev_state_root": String(
                    "7EAgMRCrBWcb3ZS6SZJ7Dm71VZ1jaBpgGiewAEvFqPT1",
                ),
                "rent_paid": String(
                    "0",
                ),
                "shard_id": Number(
                    0,
                ),
                "signature": String(
                    "ed25519:5PsEycwTdWwkZyDyzHLYZpRX3yLNq2shpTbTENo4cJaTE4HCHZhomBrURQTx2C7ttLMRDDSX9hDGjWkfoRDrLo5U",
                ),
                "tx_root": String(
                    "11111111111111111111111111111111",
                ),
                "validator_proposals": Array([]),
                "validator_reward": String(
                    "0",
                ),
            }),
        ]),
        "header": Object({
            "approvals": Array([
                String(
                    "ed25519:4HXQvSp1Yjq3t5ar8UCuPG74QegkMWQ1HMcTnipNurp5hFJruMgLHQP59W29NQ7co2iJ6nnakR1eBeFweRKVCiJk",
                ),
                String(
                    "ed25519:3tXf2x6WLprQX8u7sggM5a1MAeERVwrT9sKdVGAQxhaZLNFnVEc54KjeuwUvhAxjqxMi7dVMpsCzJHmPWny9CZjB",
                ),
                Null,
                Null,
                String(
                    "ed25519:cJrT5mN7G8wJjc3vucY2tUwpi3zmPFbhCJzYcgvw8UcjCFtkXagRPSa1x7rdMazrHEHC8sGHitDy3gkiRgKG3Fy",
                ),
                String(
                    "ed25519:4ucsVXbcKHpnQfJfepVoboRiQxkk9Q6kMAXt5HDFP5Br4rNwBqvfFh63oGEWqTY4tguq9kRzZKPWRyW6gosa9sVY",
                ),
                String(
                    "ed25519:5FKCg1fLs2y4fTrryjx8WQE4xLTWVYsaW8KjrBh5maT1WvgKVTTeWrcpoKDvFEhuo4EQUp8cGF3Kp2DcHs62Dsic",
                ),
                String(
                    "ed25519:3AJ4AqFTpjQZu41ZmvTgN6J53EmD7tJ6sFgxuvEc7Mb2A9EEPh1wFJ94ZfJRgfCbBRnYf1RVTeyUfp1af1tBpyQT",
                ),
                Null,
                String(
                    "ed25519:51Ke8WscYZnUyDpVoQeBLR8hv9GR891493CrFaR3yRsQiJfB1toSmLnJYqZyHSGJgyJFY9Sd4uwxfcSLqRf9qAv3",
                ),
                String(
                    "ed25519:5FdCnnmawAzpqizHFYoDbHbsZ2ocwP3mchGjYFmJuuAoy8dy9v798AwLLedDY6Jp2o376G85DThcp477bnbhPAwL",
                ),
                Null,
                Null,
                Null,
                String(
                    "ed25519:5URwWs7xGtrwDhnbPE2uD2mem1vARZB31XQAbChzAaZ6gVAGCiw3cBeHVNtdsjtT4hXjKD6w9UTjaCmZYjbynjnj",
                ),
                Null,
                Null,
                Null,
                String(
                    "ed25519:3uWBEoHW1uymj8d6chjRwv4FZ1C8NJS9Ffpeu3zb4GtmCX3d4aFCbG1Q8Q1pA1UoSKZLAdCwV7MnV8iiYk2szkPP",
                ),
                Null,
                Null,
            ]),
            "block_merkle_root": String(
                "BnzreFUYejuueVFCuCYqzBVtKzv2fKpw4HwATLftA3YA",
            ),
            "block_ordinal": Null,
            "challenges_result": Array([]),
            "challenges_root": String(
                "11111111111111111111111111111111",
            ),
            "chunk_headers_root": String(
                "DGkEfpsbQGbNLBUVimbhnqfyrexvMhUwWVDnmoZ6fLhZ",
            ),
            "chunk_mask": Array([
                Bool(
                    false,
                ),
            ]),
            "chunk_receipts_root": String(
                "9ETNjrt6MkwTgSVMMbpukfxRshSD1avBUUa4R4NuqwHv",
            ),
            "chunk_tx_root": String(
                "7tkzFg8RHBmMw1ncRJZCCZAizgq4rwCftTKYLce8RU8t",
            ),
            "chunks_included": Number(
                0,
            ),
            "epoch_id": String(
                "11111111111111111111111111111111",
            ),
            "epoch_sync_data_hash": Null,
            "gas_price": String(
                "5000",
            ),
            "hash": String(
                "2qjMEqiW65Jz7dhxo8PnkiMDm97UmnkorQoBKkdgAbdE",
            ),
            "height": Number(
                42376928,
            ),
            "last_ds_final_block": String(
                "11111111111111111111111111111111",
            ),
            "last_final_block": String(
                "11111111111111111111111111111111",
            ),
            "latest_protocol_version": Number(
                43,
            ),
            "next_bp_hash": String(
                "6YkAsqN3msN9w5t48AxHZMy5MEGEtcMrChHKcgZitT6d",
            ),
            "next_epoch_id": String(
                "FWJ9kR6KFWoyMoNjpLXXGHeuiy7tEY6GmoFeCA5yuc6b",
            ),
            "outcome_root": String(
                "7tkzFg8RHBmMw1ncRJZCCZAizgq4rwCftTKYLce8RU8t",
            ),
            "prev_hash": String(
                "EzrtVzJm6KLdimAqd2nCx7mAeCu3jK1bXywdxwHNvzRa",
            ),
            "prev_height": Null,
            "prev_state_root": String(
                "8iktLnBC5gmJxtbhypQedP1VdzdCXvmnoKTorr4ZxK1Y",
            ),
            "random_value": String(
                "ABofcuJHWwebRiku4BT9yodr6afNFLkmnzd3L5KfwKAN",
            ),
            "rent_paid": String(
                "0",
            ),
            "signature": String(
                "ed25519:3an7Wkc2iJ4G5Re15wF5Qzky5CvuW3torSiegkyMXX7ej9VwUkCGphEjiPcje17KMgjF5vTfTkT9TzqBwJJ2MtDK",
            ),
            "timestamp": Number(
                1617306037617521271,
            ),
            "timestamp_nanosec": String(
                "1617306037617521271",
            ),
            "total_supply": String(
                "2089646653180081825096998107194444",
            ),
            "validator_proposals": Array([]),
            "validator_reward": String(
                "0",
            ),
        }),
    }),
    "shards": Array([
        Object({
            "chunk": Null,
            "receipt_execution_outcomes": Array([]),
            "shard_id": Number(
                0,
            ),
            "state_changes": Array([]),
        }),
    ]),
})

khorolets · 2022-02-10T18:23:17Z

@frol a bit fresher one

Object({
    "block": Object({
        "author": String(
            "node1",
        ),
        "chunks": Array([
            Object({
                "balance_burnt": String(
                    "0",
                ),
                "chunk_hash": String(
                    "9kmRGeQcWRdiRqUdwHtMgwudyhzsimokgFVjEfQG2qNw",
                ),
                "encoded_length": Number(
                    287,
                ),
                "encoded_merkle_root": String(
                    "3PSVLUfjsGX6kWqCayLetH2ZYoaSSRFvHB6bRkMHJNpg",
                ),
                "gas_limit": Number(
                    1000000000000000,
                ),
                "gas_used": Number(
                    0,
                ),
                "height_created": Number(
                    82073688,
                ),
                "height_included": Number(
                    82073688,
                ),
                "outcome_root": String(
                    "11111111111111111111111111111111",
                ),
                "outgoing_receipts_root": String(
                    "8s41rye686T2ronWmFE38ji19vgeb6uPxjYMPt8y8pSV",
                ),
                "prev_block_hash": String(
                    "DmHw5ykupaGsB2xwAdp5JUYUKvA2XgB4XAKyXXiBMgmx",
                ),
                "prev_state_root": String(
                    "9Fu4sSPVChG3tHVpHw6EH1687PuPQkaRK1QJzyXcen9o",
                ),
                "rent_paid": String(
                    "0",
                ),
                "shard_id": Number(
                    0,
                ),
                "signature": String(
                    "ed25519:mTza9NufsfMirTM8eExKj72zzMdRUgN1tYMmiDd5nhNcBUMJNXzEDqo6dNbpAbbiBB7MYjHe6mQuNomMuwESRAn",
                ),
                "tx_root": String(
                    "2LYHFUsVhUzo7E9A77mFU6s4EusFSK8beAxvkoBWXDaF",
                ),
                "validator_proposals": Array([]),
                "validator_reward": String(
                    "0",
                ),
            }),
            Object({
                "balance_burnt": String(
                    "763216307602800000000",
                ),
                "chunk_hash": String(
                    "EQg4buePCSDCtQMR9dGkTZbw1rWTD1g4HEzMbJvzjgj",
                ),
                "encoded_length": Number(
                    161,
                ),
                "encoded_merkle_root": String(
                    "8cQ31ENXRVU7wPtEVZNGZ9BBLvAHMUu14M9urCju3uy8",
                ),
                "gas_limit": Number(
                    1000000000000000,
                ),
                "gas_used": Number(
                    9862287099874,
                ),
                "height_created": Number(
                    82073688,
                ),
                "height_included": Number(
                    82073688,
                ),
                "outcome_root": String(
                    "EGPnDVwJCvNPFzN86EDm9zoLJLXAQMuEDto5PmW6rn5m",
                ),
                "outgoing_receipts_root": String(
                    "EXu7nw8VVLPwvt2AsDzaJ4k3qZu2VbQHjFTH5X8KPx5v",
                ),
                "prev_block_hash": String(
                    "DmHw5ykupaGsB2xwAdp5JUYUKvA2XgB4XAKyXXiBMgmx",
                ),
                "prev_state_root": String(
                    "oaPwaoKQ1HKGmR6KuNYA7x4hxjcY9U1Wx9kngZaKhVp",
                ),
                "rent_paid": String(
                    "0",
                ),
                "shard_id": Number(
                    1,
                ),
                "signature": String(
                    "ed25519:33pVRH35Gby4SYDNSLNzxxr1t6F8EHw8ChkkCRbBmguWzVDWCSmVQdbw8SFJjduuC4zRWPNSRu5rC8s5qeEantZi",
                ),
                "tx_root": String(
                    "11111111111111111111111111111111",
                ),
                "validator_proposals": Array([]),
                "validator_reward": String(
                    "0",
                ),
            }),
            Object({
                "balance_burnt": String(
                    "12654145262112500000000",
                ),
                "chunk_hash": String(
                    "9XLALCnuDGmjuALezzcCPn6vgCQJv1ecyY5MzkpD8qLh",
                ),
                "encoded_length": Number(
                    328,
                ),
                "encoded_merkle_root": String(
                    "CeRxs3vTpwpjWRfbrprdnyNwcUgeBR65WmfToswMkwwc",
                ),
                "gas_limit": Number(
                    1000000000000000,
                ),
                "gas_used": Number(
                    176875397519684,
                ),
                "height_created": Number(
                    82073688,
                ),
                "height_included": Number(
                    82073688,
                ),
                "outcome_root": String(
                    "DGkjHBSNzLionCZoyyaaaYQeckkVb2NiJh9VF5PuR7Cx",
                ),
                "outgoing_receipts_root": String(
                    "3gxPfnudU9SFebYLn8vjGwpNKnRKt4CkmLBke85MK2Q7",
                ),
                "prev_block_hash": String(
                    "DmHw5ykupaGsB2xwAdp5JUYUKvA2XgB4XAKyXXiBMgmx",
                ),
                "prev_state_root": String(
                    "J2RzLg46dyVi2rjqkF3jFjitMEbJBUeRKYoTr1bFWgZ5",
                ),
                "rent_paid": String(
                    "0",
                ),
                "shard_id": Number(
                    2,
                ),
                "signature": String(
                    "ed25519:35M5wtLB3uBeyK3UdE3wt7H7vPmQnNqW1b9qhKKvn73B8jYwcb3SALKfbcefR87hRMX4k1nk7nWDH4E6tChDyecn",
                ),
                "tx_root": String(
                    "11111111111111111111111111111111",
                ),
                "validator_proposals": Array([]),
                "validator_reward": String(
                    "0",
                ),
            }),
            Object({
                "balance_burnt": String(
                    "2121509021969200000000",
                ),
                "chunk_hash": String(
                    "9RQeRZbKTq4xtb4inckzMTqznydrQDazRsKJCa4yoDQJ",
                ),
                "encoded_length": Number(
                    1314,
                ),
                "encoded_merkle_root": String(
                    "7Wh1h38A5B7wStaLu2GCU7Nbi9WFbfEhYh99mAQuVjPs",
                ),
                "gas_limit": Number(
                    1000000000000000,
                ),
                "gas_used": Number(
                    27184942251574,
                ),
                "height_created": Number(
                    82073688,
                ),
                "height_included": Number(
                    82073688,
                ),
                "outcome_root": String(
                    "36VcUVPDJqGhEn46Da7jDZjZSApK7HefVpSxBBnYMX4V",
                ),
                "outgoing_receipts_root": String(
                    "ABozP1kf6ZYTrGooWcDc3tWJL4XiJ3prSTTJ2jo9pW7y",
                ),
                "prev_block_hash": String(
                    "DmHw5ykupaGsB2xwAdp5JUYUKvA2XgB4XAKyXXiBMgmx",
                ),
                "prev_state_root": String(
                    "Dnq6D4vRNDtVVN4jnJTaVEJhbN1NWTKDhkeWYXR7pPwQ",
                ),
                "rent_paid": String(
                    "0",
                ),
                "shard_id": Number(
                    3,
                ),
                "signature": String(
                    "ed25519:4m3FPR1zDBVxB4kb26gwRN49iuSmjCNEQzzGLRLyvdgzTawWRR2E3DTnEkrEoUdvxo3wFzTNUZVQ1xmx6w2FUTCg",
                ),
                "tx_root": String(
                    "11111111111111111111111111111111",
                ),
                "validator_proposals": Array([]),
                "validator_reward": String(
                    "0",
                ),
            }),
        ]),
        "header": Object({
            "approvals": Array([
                String(
                    "ed25519:3wqQsRnjyuTCR4xa1gdkyg1A6pCqDT7Ps4uo1T5uZRhHJ24rEFpUV2EV7pbYfau2VC6CkWTtVNenUSNyBZn7P2Xc",
                ),
                String(
                    "ed25519:3wqQsRnjyuTCR4xa1gdkyg1A6pCqDT7Ps4uo1T5uZRhHJ24rEFpUV2EV7pbYfau2VC6CkWTtVNenUSNyBZn7P2Xc",
                ),
                String(
                    "ed25519:3wqQsRnjyuTCR4xa1gdkyg1A6pCqDT7Ps4uo1T5uZRhHJ24rEFpUV2EV7pbYfau2VC6CkWTtVNenUSNyBZn7P2Xc",
                ),
                String(
                    "ed25519:3wqQsRnjyuTCR4xa1gdkyg1A6pCqDT7Ps4uo1T5uZRhHJ24rEFpUV2EV7pbYfau2VC6CkWTtVNenUSNyBZn7P2Xc",
                ),
                Null,
                Null,
                String(
                    "ed25519:7ZNfn72BU6KQ6j22RRXwezZWy9uUTVemfq6P9VUmdbUBPQ2nPkP7zi5WmsWLuRhWiGHegko9DCJMyTZFL9PR7wM",
                ),
                Null,
                String(
                    "ed25519:3eUDdvg7deZqyT8wK6ZGwq6pvAX5Dytt9Eyp2PukVVCmKLYGRwntve8Mtbnn5BbHQ5qCZR9PXHZtXFpPnvez4oxr",
                ),
                Null,
                String(
                    "ed25519:5PNGVABWQUFPm2qkFFhPWnjwCsNMzqDxLW39PMtkRMqw3MAXN8MyxgWEghmEtCzgkBZZEDhkeS36KbR8myHziWP5",
                ),
                Null,
                Null,
                Null,
                String(
                    "ed25519:5YNTosmsrETrgs8bE7yAhXuM2gVjdqRPzfsPMxhvKGawhFRHjXmg5g3VoPL9r9xBWLq1wHArP2dtonLuTwDLawci",
                ),
                Null,
                Null,
                String(
                    "ed25519:5DgXP2REnk9L3N8xMWyKx3WYVNG7umzgsidkAGWcqw89qde1o8ZqvD4jiDfoyVpP717mJ9rANRB3aUxhatYqHzJ9",
                ),
                String(
                    "ed25519:2vnePSFsYi8mmDcXqt7WHEbmUGLut4fDQSRLKX4np7Sf6VK3z1F5v4x2VbDRtZcJovmmfKk2RBzi2n6GNSNAm74v",
                ),
                String(
                    "ed25519:2A6iLEqaewQaMKzhGQZPQ54VFMM4ApAp36de7DFEaeksPN4ZxG94gqiKBbWZgGB1J9J6DjaEJE5w3u9tAPigYez6",
                ),
                String(
                    "ed25519:5apTWsMjBWKVYZzHL3tdsBmMGKaafZAjkMisbWYtSKMkCjkZ6iEZiuKtbmULko3n55mT46grzvxeWQEDHb4axTwT",
                ),
                Null,
                String(
                    "ed25519:58LuucVUdtCJ2rLLVvo5kcKytC6RZLC9LPLxSkEkFo5HgdgTxuZemEcmnoxD8WW24ZVPRkok3w3YbUYZKUqqMhM2",
                ),
                Null,
                String(
                    "ed25519:5CDmbjXHaghTZCP7XTKFxKR3csU12ZxdHVFBaqafn7niddFwQYxnsaZ6sTtxpF7LTyvAPC4VxpehTnq9muudAQ8H",
                ),
                String(
                    "ed25519:3B4p2YxtAj5AefYm1BuiYcHpZHicgzinDCjcdzej4cQBj28yrXcN2rZecn7TLQDac3Gb7UwNnjfNyNyhBFftwF1n",
                ),
                Null,
                String(
                    "ed25519:2M2fUK1Z4Wt2DzGGMNTT37TPqbKNLx2T42rVSHJbCDVcKpfGzK6tiWuPrh9iHADcy7stR3g6KwPMnosDBV4k4AMw",
                ),
                Null,
                Null,
                Null,
                String(
                    "ed25519:F6wBdvKA2hGTQ8gFRZyjizdg3xi7kzAsZPmmQvvpvpVUc4tAff5PJ1sf1HEY2Fhnyw94g1vnBSQE1qwzqdgTAnM",
                ),
                Null,
                Null,
                Null,
                Null,
                String(
                    "ed25519:5CedpkFBbYE34Ndju8v4gocpSCJKXuXBex5WjEU3oWKTSQgSAi8UxXo6WSnXzVMKaRLWCiPSMeYpUSuMVXu9aC4m",
                ),
                Null,
                Null,
                Null,
                String(
                    "ed25519:4nn7sviHSrfqKS5jH1xxzjrEd1XsMysZwymvXeuEJmY5xJKV7t7PL9epwEbFXhM9CeDxSsc78e7QQ8UpVZxLzkC8",
                ),
                String(
                    "ed25519:4sFpT8JGkxsQ8TCvcRDd45fTq7iB6M1cu8h4V6CtWc62gXYGmPHEwrQZnsPz83Po5ZhQhvKaZrTFGGVPgpgVZbuV",
                ),
                String(
                    "ed25519:2KqLPdefDnFJaeBquke2okth98Rni2d4H4VvHvS1fwnxitApvhNqVa1pxunVrgEEGNcyomsaCLZUtoqZ8TujZkHy",
                ),
                String(
                    "ed25519:5ez1bpLXjRcr8mYoXaziDDyaVTEUiJEdtBiCrYQM2HwXZ4DkvwFgj35CFjU2ZqfxCQkUmdkLZ3HVXP6YUGZpHuPF",
                ),
                String(
                    "ed25519:4RtDGc9JknyG9vC7taj9pV3jg1ZFiMPeUBA83LFHBzPmRQNXNown5E1PVCBkSzNTtZTiCtwpe6JhVpHfMLiTEMUL",
                ),
                String(
                    "ed25519:5dH2PGAcUtAzfJ3uvcnUsxHFvBvh2ZnA3D1v7YFAxREqhnDBRcggsarxWWPhcG8dunW1dxi7xhvWWd1beQCMiiGV",
                ),
                String(
                    "ed25519:2Rkkmi3K41vyHRYxhmDxZW5azqidPJvHdNxHfiPvz9JWHDByo8JhJepK2FfCFxhhnbZQdeZUmYsSPTPWD8LQ1Q4L",
                ),
                Null,
                String(
                    "ed25519:2XENCFGU8CAtMiDqyTWpAeQi7ERQFT4ZvDBZZKfazEAdLPyskHuJ83zSEfLKsUtC41nFPtvPgJfBvP9DuBK3Si37",
                ),
                Null,
                Null,
                String(
                    "ed25519:3CtDQRRFUGYKvKTEWMKjN4XBDXZT3jjfXJXepz46MUW27rnsMKgc8QtmxwS4Vx1fLQz288oHa7F31r2LRcLX6KrT",
                ),
                Null,
                Null,
                Null,
                Null,
                String(
                    "ed25519:5iKEL4XgxKUfUzLeed8HqBnvUSCJwu9G3tHZDTp3D7EviTbd1LDsBLnuFXCGzTvczrWFcWM9vvcHDxW1hDBZq9KD",
                ),
                String(
                    "ed25519:3gYYDNFrB6Pptd6jCF4bXFZny6wUAbaJ3Uy9osuqhJxNJhvptm7J5FcyYx8KCu5Ua1iftS7YhHyQMB8wpzcWvv9A",
                ),
                Null,
                String(
                    "ed25519:3jYLQVXxRgULTciphC6TQtxDp6SB9h8QNdFbURw1bTEEGYt1vkFYTQj5R76SrHG28pGqV4YmFrJfsKnCk5g7354t",
                ),
                Null,
                Null,
                Null,
                String(
                    "ed25519:5U7BB9pPivr2JAhqpDD78kaDD5iLT1HTTcJ9AAiKYYnkjLYzzCJXCrp4cxfi6qSpAgaJdextWMfcQnwzvsAPuy19",
                ),
                Null,
                Null,
                String(
                    "ed25519:3hYbo7Hsez2j7ERXiZMHBEzHEp1GDNfNGWc3kA6LeHx6cC4szm9wuL7KN9ufPt4nafedc5XEJ4ht6J8fSzx9gXcn",
                ),
                Null,
                Null,
                String(
                    "ed25519:5beyL1iDUcXBa8ice6uxbHB2zp2Xn66QTdgquKY9EgFc5smQgUbXFHAbhx6y9qRph8VHXBemrG4VMJ5hN9183m89",
                ),
                Null,
                String(
                    "ed25519:4rF59V9YSkXbEMprD3ATYP3JcxWeJkVNHM4ZfM6ApnhqQn7q4qowEFFpLo7zNihCvoEwq9W9PGruL3tVbpzE7vim",
                ),
                Null,
                String(
                    "ed25519:4xNcD8SMKaQKKfa6btzWon2Kht1a3nmN5eCe23UGddYMfa6mrk2D1Nc8m1JMgPCwy2i3nwEn9EHnFxFw3WTDv5uh",
                ),
                String(
                    "ed25519:35WwJodi7LgvLbXGVcRCXitAfeL4YNDuKYgeMtMT2m858FF7GC7fsXJJDKnVWiH6WwrXiq2WpcCnBapmcNKiYmiY",
                ),
                String(
                    "ed25519:3nxqVBeDydpHnXDnYU8b1ENqhGFH973phS9jo3eYqyrprfpqJ4f47Fo8pZW9gLDyrguTs9gBhctn98UHH2gNnW14",
                ),
                String(
                    "ed25519:5RXDitdgnm1QgxwUfhcJyTz5tBZg7JBjeAa4MvUEs6um2Puhh3qXw5LBY7KRGKtnc5LkKMmKsRD1954vgZRQZ1AH",
                ),
                Null,
                String(
                    "ed25519:5Acm5iHoZ1TpPeU3aXMSG3MiUVcCf1YYRwvGQVHkeTavQGhjvJ7h11CDYQo5hXH1pBEuFJh4W4Q1aJde771AbsRp",
                ),
                String(
                    "ed25519:fNFwmTJqaLKZfZ5fxYjYUZH75N626UTgVSiYfyCLTq1MSG1taHzcoVjn5uuCWXKYBYpoHUChRP5UWR3C5gj8dZU",
                ),
                String(
                    "ed25519:3BYERsaqS3fGPXNeEazJQdEeKbDCVg3orXGAs2jU5HrqNgWtb4hFYq1qxxdBT3NgkcedF2MQGL2z1xRYrKVSREQp",
                ),
                String(
                    "ed25519:5QB5am4hAbgKD9g8NirdjZd7TJwWDYLp1GXkkruGnCJXVuypuanhjCvCjZhsBKEPy9Wx5untfi31GMiAmLEnr74d",
                ),
                String(
                    "ed25519:2DP2TPCY57Bic6y7T1K2f3rsAQzSLhpk9dKnnj2hGnxu4dezrZwJ9RXBpFJo8au7Qu5DA63337aBGTQkmrYWWbnU",
                ),
                String(
                    "ed25519:22JgeSLafDJYGmeswzaRBnkSP8pXDK7S5EMNiyfUQNssEUuWXBzWDCj337YTFerw9bk1XLr5t39Rocik3P5WLHm2",
                ),
                String(
                    "ed25519:351DnoAhsrdCAiMUFjSiV6v1ZPj4yutxMAK7JsLowAbYe8hJJJzB9oeGEFqyPJqJk6vVE1egPUWiyCG7tCacJ9n4",
                ),
                Null,
                Null,
                String(
                    "ed25519:LSa96nABpF8kxfWWTyz5VPj3okrhKeurxmtvVKYGWDs2BjtVVjKWEvQYHyZoUguSaWM1aCxsfLDxBQx4jWK6oC8",
                ),
            ]),
            "block_merkle_root": String(
                "8eE34U991F1cY2BmwqDKzqjLet3vqdKGCDkJf5wjMVAS",
            ),
            "block_ordinal": Number(
                37946905,
            ),
            "challenges_result": Array([]),
            "challenges_root": String(
                "11111111111111111111111111111111",
            ),
            "chunk_headers_root": String(
                "3ToZJ7nGwRyG1HSceayyR7ii599rngEjKVYTfCLYBAuu",
            ),
            "chunk_mask": Array([
                Bool(
                    true,
                ),
                Bool(
                    true,
                ),
                Bool(
                    true,
                ),
                Bool(
                    true,
                ),
            ]),
            "chunk_receipts_root": String(
                "5ZMbXN8wXy9Z9dzwxw4ngSsY3VyGQowzpG2eTaVh78Q7",
            ),
            "chunk_tx_root": String(
                "CtBJdQgvgnMAT9pczq4zX4RfhND8gpa1fJQRqchtrvtd",
            ),
            "chunks_included": Number(
                4,
            ),
            "epoch_id": String(
                "ALGL47tt455JBmJ7ro4FpW7ZABxjwkkEFBfKwvcqcpif",
            ),
            "epoch_sync_data_hash": Null,
            "gas_price": String(
                "100000000",
            ),
            "hash": String(
                "5qMRAshABScSEwNq63kse8GLmHBVJhPc4AGtw9UuYTTt",
            ),
            "height": Number(
                82073688,
            ),
            "last_ds_final_block": String(
                "DmHw5ykupaGsB2xwAdp5JUYUKvA2XgB4XAKyXXiBMgmx",
            ),
            "last_final_block": String(
                "B8U1CAW6gNnByXdssSfYKjEY9vDa9axRbea3ivs1K5XR",
            ),
            "latest_protocol_version": Number(
                51,
            ),
            "next_bp_hash": String(
                "5tuWiY1tde4cHBfc6kRkJYyNF1K5hXEjGfaXATHLrkMY",
            ),
            "next_epoch_id": String(
                "7hW4XuAaLNRDHCgVZ51M79CSUvAzubsjiWNhzHWyynxo",
            ),
            "outcome_root": String(
                "GDVDDewghwe85EsY1nAo2RL9nP2M6KisTiuCjPuru4jQ",
            ),
            "prev_hash": String(
                "DmHw5ykupaGsB2xwAdp5JUYUKvA2XgB4XAKyXXiBMgmx",
            ),
            "prev_height": Number(
                82073687,
            ),
            "prev_state_root": String(
                "J9ZAyesZZu8f9LUMR9RWK9puyTM8bC9S4h9L5CmPQ8yT",
            ),
            "random_value": String(
                "CY3acQu9cDmDDThUcDxrz5gLgwKyTi3Bvg8QkLhkRsHF",
            ),
            "rent_paid": String(
                "0",
            ),
            "signature": String(
                "ed25519:3UPLHLc1WXgaoivcC8zwqAX47yMyNRYka61zVQKQ7ZDAh482VeCi9i6qgt82guo3ZeXN5UsD3K7TheNEchpzjnCr",
            ),
            "timestamp": Number(
                1644516969713720676,
            ),
            "timestamp_nanosec": String(
                "1644516969713720676",
            ),
            "total_supply": String(
                "2180130170786047734234487799856503",
            ),
            "validator_proposals": Array([]),
            "validator_reward": String(
                "0",
            ),
        }),
    }),
    "shards": Array([
        Object({
            "chunk": Object({
                "author": String(
                    "node1",
                ),
                "header": Object({
                    "balance_burnt": String(
                        "0",
                    ),
                    "chunk_hash": String(
                        "9kmRGeQcWRdiRqUdwHtMgwudyhzsimokgFVjEfQG2qNw",
                    ),
                    "encoded_length": Number(
                        287,
                    ),
                    "encoded_merkle_root": String(
                        "3PSVLUfjsGX6kWqCayLetH2ZYoaSSRFvHB6bRkMHJNpg",
                    ),
                    "gas_limit": Number(
                        1000000000000000,
                    ),
                    "gas_used": Number(
                        0,
                    ),
                    "height_created": Number(
                        82073688,
                    ),
                    "height_included": Number(
                        82073688,
                    ),
                    "outcome_root": String(
                        "11111111111111111111111111111111",
                    ),
                    "outgoing_receipts_root": String(
                        "8s41rye686T2ronWmFE38ji19vgeb6uPxjYMPt8y8pSV",
                    ),
                    "prev_block_hash": String(
                        "DmHw5ykupaGsB2xwAdp5JUYUKvA2XgB4XAKyXXiBMgmx",
                    ),
                    "prev_state_root": String(
                        "9Fu4sSPVChG3tHVpHw6EH1687PuPQkaRK1QJzyXcen9o",
                    ),
                    "rent_paid": String(
                        "0",
                    ),
                    "shard_id": Number(
                        0,
                    ),
                    "signature": String(
                        "ed25519:mTza9NufsfMirTM8eExKj72zzMdRUgN1tYMmiDd5nhNcBUMJNXzEDqo6dNbpAbbiBB7MYjHe6mQuNomMuwESRAn",
                    ),
                    "tx_root": String(
                        "2LYHFUsVhUzo7E9A77mFU6s4EusFSK8beAxvkoBWXDaF",
                    ),
                    "validator_proposals": Array([]),
                    "validator_reward": String(
                        "0",
                    ),
                }),
                "receipts": Array([]),
                "transactions": Array([
                    Object({
                        "outcome": Object({
                            "execution_outcome": Object({
                                "block_hash": String(
                                    "5qMRAshABScSEwNq63kse8GLmHBVJhPc4AGtw9UuYTTt",
                                ),
                                "id": String(
                                    "7YeT3ywirVV2kJwRBcR4nDg1371PBUr2gbYykLrBFatL",
                                ),
                                "outcome": Object({
                                    "executor_id": String(
                                        "amberdata.testnet",
                                    ),
                                    "gas_burnt": Number(
                                        2428035032634,
                                    ),
                                    "logs": Array([]),
                                    "metadata": Object({
                                        "gas_profile": Null,
                                        "version": Number(
                                            1,
                                        ),
                                    }),
                                    "receipt_ids": Array([
                                        String(
                                            "EXwq9HqpkMTq9LRMs19c4UTvsBGkbAJPNVdiVHedk4UL",
                                        ),
                                    ]),
                                    "status": Object({
                                        "SuccessReceiptId": String(
                                            "EXwq9HqpkMTq9LRMs19c4UTvsBGkbAJPNVdiVHedk4UL",
                                        ),
                                    }),
                                    "tokens_burnt": String(
                                        "242803503263400000000",
                                    ),
                                }),
                                "proof": Array([]),
                            }),
                            "receipt": Null,
                        }),
                        "transaction": Object({
                            "actions": Array([
                                Object({
                                    "FunctionCall": Object({
                                        "args": String(
                                            "eyJwYWlyIjoiQlRDIC8gVVNEIiwicHJpY2UiOiI0NTE1OTE1MDc2OSJ9",
                                        ),
                                        "deposit": String(
                                            "300800000000000000000000",
                                        ),
                                        "gas": Number(
                                            300000000000000,
                                        ),
                                        "method_name": String(
                                            "push_data",
                                        ),
                                    }),
                                }),
                            ]),
                            "hash": String(
                                "7YeT3ywirVV2kJwRBcR4nDg1371PBUr2gbYykLrBFatL",
                            ),
                            "nonce": Number(
                                78052809193894,
                            ),
                            "public_key": String(
                                "ed25519:HakbHwGjynqqEJ3GMCJwZJAUQenqQMDfA8b4sPQMBtnz",
                            ),
                            "receiver_id": String(
                                "fpo3.franklinwaller2.testnet",
                            ),
                            "signature": String(
                                "ed25519:5iskyo6yUX3AstTb6zHStVNtaG1PP89uZYiQrsArmkf66nedT9w9A5WGnyqfUEVqrfsqmAEEjnTWHheT65XnR1KY",
                            ),
                            "signer_id": String(
                                "amberdata.testnet",
                            ),
                        }),
                    }),
                ]),
            }),
            "receipt_execution_outcomes": Array([]),
            "shard_id": Number(
                0,
            ),
            "state_changes": Array([
                Object({
                    "cause": Object({
                        "tx_hash": String(
                            "7YeT3ywirVV2kJwRBcR4nDg1371PBUr2gbYykLrBFatL",
                        ),
                        "type": String(
                            "transaction_processing",
                        ),
                    }),
                    "change": Object({
                        "account_id": String(
                            "amberdata.testnet",
                        ),
                        "amount": String(
                            "57236642182111505101493774",
                        ),
                        "code_hash": String(
                            "11111111111111111111111111111111",
                        ),
                        "locked": String(
                            "0",
                        ),
                        "storage_paid_at": Number(
                            0,
                        ),
                        "storage_usage": Number(
                            346,
                        ),
                    }),
                    "type": String(
                        "account_update",
                    ),
                }),
                Object({
                    "cause": Object({
                        "tx_hash": String(
                            "7YeT3ywirVV2kJwRBcR4nDg1371PBUr2gbYykLrBFatL",
                        ),
                        "type": String(
                            "transaction_processing",
                        ),
                    }),
                    "change": Object({
                        "access_key": Object({
                            "nonce": Number(
                                78052809193894,
                            ),
                            "permission": String(
                                "FullAccess",
                            ),
                        }),
                        "account_id": String(
                            "amberdata.testnet",
                        ),
                        "public_key": String(
                            "ed25519:HakbHwGjynqqEJ3GMCJwZJAUQenqQMDfA8b4sPQMBtnz",
                        ),
                    }),
                    "type": String(
                        "access_key_update",
                    ),
                }),
            ]),
        }),
    ]),
})

khorolets · 2022-02-11T13:38:00Z

@frol are we waiting for the merging of the PR on the nearcore side before closing this issue or we want to perform some action points?

frol · 2022-02-11T14:00:27Z

@khorolets I suggest we also drop the duplicate chunks from the block structure.

khorolets · 2022-02-14T15:39:08Z

During the 1-on-1 we've decided to leave ChunkHeaderView duplicate.

@frol should we close this issue?

frol · 2022-02-14T15:51:26Z

Yes, we decided that this is not a huge deal given that I was only talking about the framework consumer side of the StreamerMessage (I did not suggest to cut the chunk headers from the block.json file on S3)

khorolets self-assigned this Jan 31, 2022

frol added this to the MVP Ready for Release milestone Feb 9, 2022

frol closed this as completed Feb 14, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Revise `near_indexer::StreamerMessage` struct #1

Revise `near_indexer::StreamerMessage` struct #1

khorolets commented Jan 31, 2022

khorolets commented Jan 31, 2022

khorolets commented Feb 8, 2022

khorolets commented Feb 10, 2022

khorolets commented Feb 10, 2022

khorolets commented Feb 11, 2022

frol commented Feb 11, 2022

khorolets commented Feb 14, 2022

frol commented Feb 14, 2022

Revise near_indexer::StreamerMessage struct #1

Revise near_indexer::StreamerMessage struct #1

Comments

khorolets commented Jan 31, 2022

khorolets commented Jan 31, 2022

khorolets commented Feb 8, 2022

khorolets commented Feb 10, 2022

khorolets commented Feb 10, 2022

khorolets commented Feb 11, 2022

frol commented Feb 11, 2022

khorolets commented Feb 14, 2022

frol commented Feb 14, 2022

Revise `near_indexer::StreamerMessage` struct #1

Revise `near_indexer::StreamerMessage` struct #1