add submit mutation benchmark

[jingchen2222]

Add benchmark for submit mutation request.

[jingchen2222]

How to use the submit mutation performance

The [submit_mutation_benchmark.rs] sends submit mutation with the given QPS and payload size.

pre-requisites (option)

vi ~/.tendermint/config/config.toml

set prometheus = true to enable metric in tendermint

[instrumentation]

# When true, Prometheus metrics are served under /metrics on
# PrometheusListenAddr.
# Check out the documentation for the list of available metrics.
prometheus = true

# Address to listen for Prometheus collector(s) connections
prometheus_listen_addr = ":26660"

step 1: build and start localnet

cargo build --release
sh start_localnet.sh release >tm.log 2>&1 &

step 2: configure the qps and payload

for qps in [100, 200, 400, 600].iter() {
    

run_submit_mutation(qps, 1).await;

step 3: run the benchmark to submit mutation

cargo bench --package db3-sdk --bench submit_mutation_benchmark

step 4: keep an eye on the tendermint_mempool_size via http://127.0.0.1:26660/ to see if the mempool exceed limit (5000 by default)

Report

Hardware & Software

Mutation Data

• KV Pattern:

key: bm_submit_mutation_key_${pid}${ts}${index}
value: bm_submit_mutation_value_${pid}${ts}${index}

• KV Example:
  key: "bm_submit_mutation_key_46103_1673927665515693000_99"
  value: "bm_submit_mutation_value_46103_1673927665515693000_99"

let mutation = Mutation {
            ns: ns_vec.clone(),
            kv_pairs,
            nonce,
            chain_id: ChainId::MainNet.into(),
            chain_role: ChainRole::StorageShardChain.into(),
            gas_price: None,
            gas: 10,
        };

Group ID	KvPairs Lenght	Bytes per Mutation	MAX QPS
1	1	232 bytes / mutation	1600
2	10	1204 bytes/ mutation	800
3	20	2304 bytes/ mutation	400

Assumption

• The maximum throughput between: 371200 ~ 921600
• throughput can be improved by batch writing

[codecov-commenter]

Codecov Report

Merging #279 (6ae8310) into main (521a847) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main     #279   +/-   ##
=======================================
  Coverage   54.86%   54.86%           
=======================================
  Files          41       41           
  Lines        3820     3820           
=======================================
  Hits         2096     2096           
  Misses       1724     1724           

Flag	Coverage Δ
rust	54.86% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
src/base/src/test_base.rs	100.00% <100.00%> (ø)

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[imotai]

2000 qps in client with 1 KvPairs Length, 956 qps in tendermint

I[2023-01-17|17:17:40.725] finalizing commit of block                   module=consensus height=36 hash=300DB3C5EE061739CBBEDE5F71DAC690CF4F1984081D12A46F9CE93D51B00423 root=88BFBDA0EB408CC6B6988200C62A4A607D82C0F21E170751EEDC770FC039CD61 num_txs=956
I[2023-01-17|17:17:41.071] executed block                               module=state height=36 num_valid_txs=956 num_invalid_txs=0
I[2023-01-17|17:17:41.926] committed state                              module=state height=36 num_txs=956 app_hash=80758DD01A642D4165E19F963F00553BBE8DC5DA32233AFC71D939A8F97B63F1
I[2023-01-17|17:17:41.950] Timed out                                    module=consensus dur=-224.449502ms height=37 round=0 step=RoundStepNewHeight

module=consensus dur=-224.449502ms
the time used to produce block has exceeded 1s. So we can conclude that 1000 qps is the upper limitation

[imotai]

LGTM