block-generator: conduit performance benchmark scenarios. (#5532)

tools/block-generator/README.md

@@ -2,6 +2,27 @@
 
 This tool is used for testing Conduit import performance. It does this by generating synthetic blocks which are sent by mocking the Algod REST API endpoints that Conduit uses.
 
+## Benchmark Scenarios
+
+Several scenarios were designed to mimic different block traffic patterns. Scenarios can be used to test the same traffic across multiple versions of software. Each benchmark is run twice. Once with blocks containing 25000 transactions, and once with blocks containing 50000 transactions.
+
+### Organic Traffic
+
+Simulate the current mainnet traffic pattern. Approximately:
+* 15% payment transactions
+* 10% application transactions
+* 75% asset transactions
+
+With current tooling, the app transactions use boxes much more frequently than current mainnet traffic.
+
+### Payment Test (best case TPS)
+
+Blocks are entirely made up of payments. Most payments are transfers between existing accounts.
+
+### Stress Test (worst case TPS)
+
+Blocks are heavily weighted towards creating applications and boxes. This means a lot of data is being written which should translate to lower TPS.
+
 ## Scenario Configuration
 
 Block generator uses a YAML config file to describe the composition of each randomly generated block. There are three levels of configuration:
@@ -84,7 +105,7 @@ Flags:
   -h, --help            help for daemon
   -p, --port uint       Port to start the server at. (default 4010)
 ```
-  
+
 ### runner
 
 The runner mode is well suited for running the same set of tests consistently across many scenarios and for different releases. The runner mode automates this process by starting the **daemon** with many different configurations, managing a postgres database, and running a separate Conduit process configured to use them.
@@ -178,7 +199,7 @@ the `go-algorand` repo, and:
 Then you can execute the following command to run the scenario:
 
 ```sh
-./run_runner.sh ./conduit scenario.yml 
+./run_runner.sh ./conduit scenario.yml
 ```
 
 ### Scenario Report

tools/block-generator/scenarios/benchmarks/organic.25000.yml

@@ -0,0 +1,29 @@
+name: "Organic (25000)"
+genesis_accounts: 10000
+genesis_account_balance: 1000000000000
+tx_per_block: 25000
+
+# transaction distribution
+tx_pay_fraction: 0.05
+tx_asset_fraction: 0.75
+tx_app_fraction: 0.20
+
+# payment config
+pay_acct_create_fraction: 0.10
+pay_xfer_fraction: 0.90
+
+# asset config
+asset_create_fraction: 0.001
+asset_optin_fraction: 0.1
+asset_close_fraction: 0.05
+asset_xfer_fraction: 0.849
+asset_delete_fraction: 0
+
+# app kind config
+app_boxes_fraction: 1.0
+app_swap_fraction: 0.0
+
+# app boxes config
+app_boxes_create_fraction: 0.01
+app_boxes_optin_fraction: 0.1
+app_boxes_call_fraction: 0.89

tools/block-generator/scenarios/benchmarks/organic.50000.yml

@@ -0,0 +1,29 @@
+name: "Organic (50000)"
+genesis_accounts: 10000
+genesis_account_balance: 1000000000000
+tx_per_block: 50000
+
+# transaction distribution
+tx_pay_fraction: 0.05
+tx_asset_fraction: 0.75
+tx_app_fraction: 0.20
+
+# payment config
+pay_acct_create_fraction: 0.10
+pay_xfer_fraction: 0.90
+
+# asset config
+asset_create_fraction: 0.001
+asset_optin_fraction: 0.1
+asset_close_fraction: 0.05
+asset_xfer_fraction: 0.849
+asset_delete_fraction: 0
+
+# app kind config
+app_boxes_fraction: 1.0
+app_swap_fraction: 0.0
+
+# app boxes config
+app_boxes_create_fraction: 0.01
+app_boxes_optin_fraction: 0.1
+app_boxes_call_fraction: 0.89

tools/block-generator/scenarios/benchmarks/payment.25000.yml

@@ -0,0 +1,11 @@
+name: "Max TPS (25000)"
+genesis_accounts: 10000
+genesis_account_balance: 1000000000000
+tx_per_block: 25000
+
+# transaction distribution
+tx_pay_fraction: 1.0
+
+# payment config
+pay_acct_create_fraction: 0.02
+pay_xfer_fraction: 0.98

tools/block-generator/scenarios/benchmarks/payment.50000.yml

@@ -0,0 +1,11 @@
+name: "Max TPS (50000)"
+genesis_accounts: 10000
+genesis_account_balance: 1000000000000
+tx_per_block: 50000
+
+# transaction distribution
+tx_pay_fraction: 1.0
+
+# payment config
+pay_acct_create_fraction: 0.02
+pay_xfer_fraction: 0.98

tools/block-generator/scenarios/benchmarks/stress.25000.yml

@@ -0,0 +1,28 @@
+name: "Stress (25000)"
+genesis_accounts: 10000
+genesis_account_balance: 1000000000000
+tx_per_block: 25000
+
+# transaction distribution
+tx_pay_fraction: 0.05
+tx_app_fraction: 0.95
+
+# payment config
+pay_acct_create_fraction: 1.00
+pay_xfer_fraction: 0.00
+
+# asset config
+asset_create_fraction: 0.001
+asset_optin_fraction: 0.1
+asset_close_fraction: 0.05
+asset_xfer_fraction: 0.849
+asset_delete_fraction: 0
+
+# app kind config
+app_boxes_fraction: 1.0
+app_swap_fraction: 0.0
+
+# app boxes config
+app_boxes_create_fraction: 0.5
+app_boxes_optin_fraction: 0.5
+app_boxes_call_fraction: 0.0

tools/block-generator/scenarios/benchmarks/stress.50000.yml

@@ -0,0 +1,28 @@
+name: "Stress (50000)"
+genesis_accounts: 10000
+genesis_account_balance: 1000000000000
+tx_per_block: 50000
+
+# transaction distribution
+tx_pay_fraction: 0.05
+tx_app_fraction: 0.95
+
+# payment config
+pay_acct_create_fraction: 1.00
+pay_xfer_fraction: 0.00
+
+# asset config
+asset_create_fraction: 0.001
+asset_optin_fraction: 0.1
+asset_close_fraction: 0.05
+asset_xfer_fraction: 0.849
+asset_delete_fraction: 0
+
+# app kind config
+app_boxes_fraction: 1.0
+app_swap_fraction: 0.0
+
+# app boxes config
+app_boxes_create_fraction: 0.5
+app_boxes_optin_fraction: 0.5
+app_boxes_call_fraction: 0.0