feat: trie-iteration-benchmark - full and shallow iteration

[wacban]

This is step 1 of the following plan to benchmark how long does it take to perform shallow trie iteration. For more details about the goal please see #9101.

1. Add a neard subcommand that does full trie iteration.
2. Extend it to actually measure and report the time it takes.
3. Extend it with shallow iteration.

Once all three steps are done it will be possible to benchmark the full iteration, the shallow iteration and compare the time it takes for each.

[robin-near]

Nice!! Looking forward to what numbers you get! :D

[Longarithm]

We already have a bit of code for benchmarking, even with some results:

nearcore/nearcore/benches/store.rs

Line 23 in e32db71

fn read_trie_items(bench: &mut Bencher, shard_id: usize, mode: Mode) {

[wacban]

example execution and output for a tiny localnet setup:

cargo build && ./target/debug/neard --home ~/.near/localnet/node0 view-state trie-iteration-benchmark

% cargo build && ./target/debug/neard --home ~/.near/localnet/node0 view-state trie-iteration-benchmark
shard uid s0.v0
Account AccountId("near")
Account AccountId("node0")
AccessKey AccountId("near")
AccessKey AccountId("node0")

[wacban]

We already have a bit of code for benchmarking, even with some results:

nearcore/nearcore/benches/store.rs

Line 23 in e32db71

fn read_trie_items(bench: &mut Bencher, shard_id: usize, mode: Mode) {

Thanks @Longarithm, this is pretty cool and similar to what I'm doing. I considered migrating to the same framework but I'd rather keep my benchmark as a neard subcommand so that I can easily run in with just neard available without having to rely on full rust setup.

[wacban]

I added some more code to this PR:

• prune condition in TrieIterator that will allow me to prune the trie once we reach a key nibble that fully describes an account (aka shallow iteration)
• some unit tests for the trie iteration and the trie iteration with pruning

Let me know if you'd rather review this separately or if you prefer to wait for the full shallow iteration implementation. I'm happy to add it here or to move it to separate PR if it's easier to review this way.

[wacban]

Here are some results for full trie iteration - without pruning - on a mainnet db, with neard stopped.

TLDR - shard 3 is a killjoy
shard 0 - 1.20 hour
shard 1 - 0.94 hour
shard 2 - 0.82 hour
shard 3 - 12.02 hours

shard uid s0.v1 benchmark starting
shard uid s0.v1 benchmark finished
node count  32921385
error count 0
time        4448.119190764s
shard uid s1.v1 benchmark starting
shard uid s1.v1 benchmark finished
node count  37058340
error count 0
time        3368.931722282s
shard uid s2.v1 benchmark starting
shard uid s2.v1 benchmark finished
node count  26542770
error count 0
time        2949.398905713s
shard uid s3.v1 benchmark starting
shard uid s3.v1 benchmark finished
node count  78845856
error count 0
time        43271.077722138s

[wacban]

results for shallow iteration:

shard id    0 benchmark starting
node count  15265615
error count 0
time        2736.587123447s
stats
TrieIterationBenchmarkStats {
    visited_map: {
        "ACCOUNT": 15260055,
        "CONTRACT_CODE": 5559,
        "DELAYED_RECEIPT_INDICES": 1,
    },
    pruned_map: {
        "CONTRACT DATA": 5036,
        "ACCESS KEY": 15257777,
    },
}
shard id    0 benchmark finished


shard id    1 benchmark starting
node count  3
error count 0
time        31.245877ms
stats
TrieIterationBenchmarkStats {
    visited_map: {
        "ACCOUNT": 1,
        "CONTRACT_CODE": 1,
        "DELAYED_RECEIPT_INDICES": 1,
    },
    pruned_map: {
        "CONTRACT DATA": 1,
        "ACCESS KEY": 1,
    },
}
shard id    1 benchmark finished

shard id    2 benchmark starting
node count  8034894
error count 0
time        1731.610757799s
stats
TrieIterationBenchmarkStats {
    visited_map: {
        "CONTRACT_CODE": 10138,
        "DELAYED_RECEIPT_INDICES": 1,
        "ACCOUNT": 8024755,
    },
    pruned_map: {
        "ACCESS KEY": 8022562,
        "CONTRACT DATA": 9129,
    },
}
shard id    2 benchmark finished


shard id    3 benchmark starting
node count  1749199
error count 0
time        2305.28088897s
stats
TrieIterationBenchmarkStats {
    visited_map: {
        "CONTRACT_CODE": 13134,
        "DELAYED_RECEIPT_INDICES": 1,
        "ACCOUNT": 1736064,
    },
    pruned_map: {
        "CONTRACT DATA": 11819,
        "ACCESS KEY": 1734192,
    },
}
shard id    3 benchmark finished

[robin-near]

Interesting! I'm surprised that for shard 0 and 2 the improvement is less than a factor of 2. For shard 0 for example, roughly half of the nodes are at or above the account level. I suppose this makes sense because your stats show that simple wallets vastly outnumber smart contracts.