Benchmark note inclusion path changes

[Mirko-von-Leipzig]

See #1097 (review).

#1087 changed storage of note inclusion path from a full to a sparse merkle path. Use the stress-test benchmark tool to compare the note storage changes before and after #1087.

Unfortunately the stress-test is currently broken #1110. The fix will likely need to be backported to before #1087 in order to compare them.

The very original stress test benchmark is #657 (comment) but ideally we would get a proper before/after comparison.

[SantiagoPittella]

I ran the benchmarks in the commit of the merge with the note inclusion path changes ( 25387d2 ) and the commit right before that one ( 7dcbf64 ) which is the one that adds the network verification via the commitment of the genesis.

The notes table size was reduced, from 78.5 MB to 49.1 MB for the whole table, and from 0.8 to 0.5 KB per entry in the table.

The command that I used is the one from the stress-test README:

miden-node-stress-test seed-store --data-directory data --num-accounts 100000 --public-accounts-percentage 50

WITH the SMT for inclusion proofs

Block metrics:
Note: Each block contains 256 transactions (16 batches * 16 transactions).
The first transaction sends assets from the faucet to 255 accounts, and the remaining 255 transactions consume notes created in the previous block.
Block #    Insert Time (ms)     Get Block Inputs Time (ms)     Get Batch Inputs Time (ms)     Block Size (KB)      DB Size (MB)
---------------------------------------------------------------------------------------------------------------------------------------
0          22                   1                              0                              375.6                0.3
50         188                  9                              1                              473.6                23.9
100        200                  10                             1                              473.6                40.7
150        209                  11                             1                              473.6                58.3
200        219                  11                             1                              473.6                75.4
250        225                  12                             1                              473.6                92.0
300        228                  12                             1                              473.6                108.6
350        234                  12                             1                              473.6                125.1

Database stats:
Note: Database contains 100215 accounts and 100215 notes across all blocks.
Table                               Size (MB)       KB/Entry
----------------------------------------------------------------------
accounts                            26.1            0.3
account_deltas                      1.3             0.0
account_fungible_asset_deltas       2.2             0.0
account_non_fungible_asset_updates  0.0             -
account_storage_map_updates         0.0             -
account_storage_slot_updates        3.0             0.1
block_headers                       0.1             0.3
notes                               49.1            0.5
note_scripts                        0.0             8.0
nullifiers                          4.6             0.0
transactions                        6.0             0.1

Index stats:
Index                               Size (MB)
----------------------------------------------------------------------
idx_accounts_network_prefix         0.0
idx_notes_note_id                   4.4
idx_notes_sender                    2.9
idx_notes_tag                       1.8
idx_notes_nullifier                 4.4
idx_unconsumed_network_notes        1.1
idx_nullifiers_prefix               4.3
idx_nullifiers_block_num            4.2
idx_transactions_account_id         5.7
idx_transactions_block_num          4.2

WITHOUT the SMT:

Block metrics:
Note: Each block contains 256 transactions (16 batches * 16 transactions).
The first transaction sends assets from the faucet to 255 accounts, and the remaining 255 transactions consume notes created in the previous block.
Block #    Insert Time (ms)     Get Block Inputs Time (ms)     Get Batch Inputs Time (ms)     Block Size (KB)      DB Size (MB)
---------------------------------------------------------------------------------------------------------------------------------------
0          24                   1                              1                              375.6                0.4
50         190                  9                              1                              473.6                27.9
100        202                  10                             1                              473.6                48.3
150        211                  11                             1                              473.6                69.6
200        228                  11                             1                              473.6                90.3
250        226                  12                             1                              473.6                110.9
300        232                  12                             1                              473.6                131.2
350        235                  12                             1                              473.6                151.5

Database stats:
Note: Database contains 100215 accounts and 100215 notes across all blocks.
Table                               Size (MB)       KB/Entry
----------------------------------------------------------------------
accounts                            26.1            0.3
account_deltas                      1.3             0.0
account_fungible_asset_deltas       2.2             0.0
account_non_fungible_asset_updates  0.0             -
account_storage_map_updates         0.0             -
account_storage_slot_updates        3.0             0.1
block_headers                       0.1             0.3
notes                               78.5            0.8
note_scripts                        0.0             8.0
nullifiers                          4.6             0.0
transactions                        6.0             0.1

Index stats:
Index                               Size (MB)
----------------------------------------------------------------------
idx_accounts_network_prefix         0.0
idx_notes_note_id                   4.3
idx_notes_sender                    2.9
idx_notes_tag                       1.8
idx_notes_nullifier                 4.4
idx_unconsumed_network_notes        1.1
idx_nullifiers_prefix               4.4
idx_nullifiers_block_num            4.2
idx_transactions_account_id         5.7
idx_transactions_block_num          4.2

[Mirko-von-Leipzig]

I'll close this as completed; we can re-open if we think of anything.