Implement Gomu Gomu No Gatling Benchmarking Tool

[AbdelStark]

New Feature Request: Implement "Gomu Gomu No Gatling" Benchmarking Tool

Description

We aim to introduce a new benchmarking tool named "Gomu Gomu No Gatling" for evaluating Starknet sequencers. This tool will primarily focus on calculating the TPS (transactions per second) and potentially other metrics, such as Cairo steps per second, at a later stage.

Details

Functionality

1. The tool will accept a configuration file that lays out the various steps and actions for the benchmark.
2. It'll have a setup phase where it establishes the necessary environment for running the benchmarks.
   • Activities, such as account creation, will be handled in this phase.
3. All operations will be performed using RPC calls on the Sequencer.
4. The contracts used must be compatible with the latest version of Cairo.
5. A performance report will be generated in JSON format, containing:
   • TPS metrics for different operations
   • Average TPS, median TPS, stats for the entire test duration
   • TPS data for the last 'X' blocks, with 'X' being adjustable.
6. There will be a validation phase post-benchmark to ensure all transactions have been successfully incorporated into the blocks.

Operations Scope

For the current scope, the operations that we'll be focusing on are:

• ERC20 token transfers
• ERC721 NFT mints

Sample Configuration

The configuration file will be structured as follows:

rpc:
  url: "https://sharingan.madara.zone"

simulation:
  fail_fast: true
  setup:
    create_accounts:
      num_accounts: 10

deployer:
  address: "0x0000000000000000000000000000000000000000000000000000000000000001"
  signing_key: "0x0"
  salt: 1

Requirements

1. The tool should be efficient and lightweight.
2. Comprehensive error-handling mechanisms must be in place.
3. Accommodate future metrics expansions without significant refactoring.
4. Ensure compatibility with the latest versions of Starknet sequencers and Cairo.
5. Must be integrated in Madara benchmark workflow ([Madara current benchmarking solution](https://github.com/keep-starknet-strange/madara/tree/main/benchmarking, Madara performance benchmark page)
6. Must be usable as standalone binary or as a Rust library

@d-roak will be following and reviewing this mission.

[github-actions]

There hasn't been any activity on this issue recently, and in order to prioritize active issues, it will be marked as stale.
Please make sure to update to the latest version and check if that solves the issue. Let us know if that works for you by leaving a 👍
Because this issue is marked as stale, it will be closed and locked in 7 days if no further activity occurs.
Thank you for your contributions!

[d-roak]

Closed as completed at #3 and updated at #20