Gomu Gomu no Gatling

Blazing fast tool to benchmark Starknet sequencers 🦀.

Installation

From source

git clone https://github.com/keep-starknet-strange/gomu-gomu-no-gatling
cd gomu-gomu-no-gatling
cargo install --path .

From crates.io

cargo install --locked gatling

Run debug

RUST_LOG=debug cargo run -- shoot -c config/default.yaml

Usage

gatling --help

For Katana, currently you need to increase the DEFAULT_PREFUNDED_ACCOUNT_BALANCE in constants to 0xffffffffffffffffffffffffffffffff and run the node with flag --no-validate.

Configuration

Gomu gomu's configuration is specified as a yaml file. You can find example configurations under the config folder.

As it uses the config crate under the hood, the configuration could be specified as any other file type such as TOML or JSON.

The configuration is defined by the following spec

rpc
- url: Starknet RPC url, should be compliant with the specification
setup

v0 and v1 CAN'T be specified at the same time

erc20_contract: ERC20 contract used to benchmark transfers
- v0: Path to Cairo Zero contract artifact
- v1:
  - path: Path to Cairo contract sierra artifact
  - casm_path: Path to Cairo contract casm artifact
- erc721_contract: ERC721 contract used to benchmark mints ...
- account_contract: Account contract used to send transactions ...
- fee_token_address: Contract address of the fee token on the target chain
- num_accounts: Number of accounts sending transactions
run
- num_erc20_transfers: Number of ERC20 transfer transactions
- num_erc721_mints: Number of ERC721 mint transactions
- concurrency: How many transactions to do simultaneously
report
- num_blocks: Number of last blocks to take into account in the report
- output_location: Path to the file where to save the reports
deployer
- salt: Salt used to compute deployment addresses
- address: Address of the deployer account (should be pre-funded)
- signing_key: Private key of the deployer signer

Run a load test

gatling shoot -c config/default.yaml

Output

The main output of gomu gomu is the report output location specified in specified in the configuration file.

At the end of all benchmarks gomu gomu will collect the results into a single json file with the following structure:

users: The amount of goose users used to do the benchmarks, changed by concurrency
all_bench_report: A report over all benchmarks done, has a portion of metrics that benches has
benches: A array of reports for all benchmarks
- name: The name of the benchmark
- amount: How many times this benchmark was ran
- metrics: Metrics over the whole benchmark
  - name: The name of the metric
  - unit: The unit of the metric, empty when there is no unit
  - value: The metrics value, a number
    - For floats, Infinite and NaN are not JSON numbers and thus will be turned into null
    - Values gotten from submission time are calculated from the latency to add a new transaction to the node
    - Values gotten from verification time are calculated from the latency to get the transaction receipt after the transactions have been processed
- last_x_blocks_metrics: Metrics over the last blocks of the benchmark
  - num_blocks: The amount of last transactions that were measured
  - metrics: An array of metrics
extra: Extra information for this run

Gomu gomu will also display into the console information about each step in the benchmark.

Resources

Gomu Gomu is originally inspired from Flood
(Aptos load-testing tool)[https://github.com/aptos-labs/aptos-multi-region-bench]
(Starknet RPC specs)[https://github.com/starkware-libs/starknet-specs/blob/master/api/starknet_api_openrpc.json]