docs(qa/v1): Report on saturation point and 200-nodes test (#2667)

Closes #2461, #2513 This report includes results for tests with and without latency emulation. [Rendered](https://github.com/cometbft/cometbft/blob/hvanz/qa-v1-report-2461/docs/references/qa/CometBFT-QA-v1.md) --- #### PR checklist - [ ] Tests written/updated - [ ] Changelog entry added in `.changelog` (we use [unclog](https://github.com/informalsystems/unclog) to manage our changelog) - [ ] Updated relevant documentation (`docs/` or `spec/`) and code comments - [X] Title follows the [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/) spec --------- Co-authored-by: Daniel <daniel.cason@informal.systems> Co-authored-by: Sergio Mena <sergio@informal.systems> (cherry picked from commit 53e6127) # Conflicts: # scripts/qa/reporting/prometheus_plotter.py
cometbft · Apr 10, 2024 · 20cab9e · 20cab9e
1 parent 4f4faec
commit 20cab9e
Show file tree

Hide file tree

Showing 50 changed files with 526 additions and 1 deletion.
diff --git a/docs/references/qa/CometBFT-QA-v1.md b/docs/references/qa/CometBFT-QA-v1.md
diff --git a/docs/references/qa/README.md b/docs/references/qa/README.md
@@ -24,3 +24,4 @@ The following releases have undergone the Quality Assurance process, and the cor
 * [TM v0.37.x](TMCore-QA-37.md) - Tested prior to releasing TM v0.37.x, using TM v0.34.x results as baseline.
 * [v0.37.x](CometBFT-QA-37.md) - Tested on CometBFT v0.37.0-alpha3, using TM v0.37.x results as baseline.
 * [v0.38.x](CometBFT-QA-38.md) - Tested on v0.38.0-alpha.2, using v0.37.x results as baseline.
+* [v1.x](CometBFT-QA-v1.md) - Tested on v1.0.0-alpha.2, using v0.38.x results as baseline.
diff --git a/docs/references/qa/img38/200nodes/avg_mempool_size_ylim.png b/docs/references/qa/img38/200nodes/avg_mempool_size_ylim.png
diff --git a/docs/references/qa/img38/200nodes/block_size_bytes.png b/docs/references/qa/img38/200nodes/block_size_bytes.png
diff --git a/docs/references/qa/img38/200nodes/memory_ylim.png b/docs/references/qa/img38/200nodes/memory_ylim.png
diff --git a/docs/references/qa/img38/200nodes/mempool_size_max.png b/docs/references/qa/img38/200nodes/mempool_size_max.png
diff --git a/docs/references/qa/img38/200nodes/rounds_ylim.png b/docs/references/qa/img38/200nodes/rounds_ylim.png
diff --git a/docs/references/qa/img38/200nodes/total_txs_rate_ylim.png b/docs/references/qa/img38/200nodes/total_txs_rate_ylim.png
diff --git a/...rences/qa/imgs/v1/200nodes/latencies/e_8e4e1e81-c171-4879-b86f-bce96ee2e861.png b/...rences/qa/imgs/v1/200nodes/latencies/e_8e4e1e81-c171-4879-b86f-bce96ee2e861.png
diff --git a/docs/references/qa/imgs/v1/200nodes/metrics/avg_cpu.png b/docs/references/qa/imgs/v1/200nodes/metrics/avg_cpu.png
diff --git a/docs/references/qa/imgs/v1/200nodes/metrics/avg_memory.png b/docs/references/qa/imgs/v1/200nodes/metrics/avg_memory.png
diff --git a/docs/references/qa/imgs/v1/200nodes/metrics/avg_mempool_size.png b/docs/references/qa/imgs/v1/200nodes/metrics/avg_mempool_size.png
diff --git a/docs/references/qa/imgs/v1/200nodes/metrics/block_rate.png b/docs/references/qa/imgs/v1/200nodes/metrics/block_rate.png
diff --git a/docs/references/qa/imgs/v1/200nodes/metrics/block_rate_regular.png b/docs/references/qa/imgs/v1/200nodes/metrics/block_rate_regular.png
diff --git a/docs/references/qa/imgs/v1/200nodes/metrics/block_size_bytes.png b/docs/references/qa/imgs/v1/200nodes/metrics/block_size_bytes.png
diff --git a/docs/references/qa/imgs/v1/200nodes/metrics/blocks.png b/docs/references/qa/imgs/v1/200nodes/metrics/blocks.png
diff --git a/docs/references/qa/imgs/v1/200nodes/metrics/cpu.png b/docs/references/qa/imgs/v1/200nodes/metrics/cpu.png
diff --git a/docs/references/qa/imgs/v1/200nodes/metrics/memory.png b/docs/references/qa/imgs/v1/200nodes/metrics/memory.png
diff --git a/docs/references/qa/imgs/v1/200nodes/metrics/mempool_size.png b/docs/references/qa/imgs/v1/200nodes/metrics/mempool_size.png
diff --git a/docs/references/qa/imgs/v1/200nodes/metrics/mempool_size_max.png b/docs/references/qa/imgs/v1/200nodes/metrics/mempool_size_max.png
diff --git a/docs/references/qa/imgs/v1/200nodes/metrics/peers.png b/docs/references/qa/imgs/v1/200nodes/metrics/peers.png
diff --git a/docs/references/qa/imgs/v1/200nodes/metrics/rounds.png b/docs/references/qa/imgs/v1/200nodes/metrics/rounds.png
diff --git a/docs/references/qa/imgs/v1/200nodes/metrics/total_txs.png b/docs/references/qa/imgs/v1/200nodes/metrics/total_txs.png
diff --git a/docs/references/qa/imgs/v1/200nodes/metrics/total_txs_rate.png b/docs/references/qa/imgs/v1/200nodes/metrics/total_txs_rate.png
diff --git a/docs/references/qa/imgs/v1/200nodes/metrics/total_txs_rate_regular.png b/docs/references/qa/imgs/v1/200nodes/metrics/total_txs_rate_regular.png
diff --git a/docs/references/qa/imgs/v1/200nodes/v1_report_tabbed.txt b/docs/references/qa/imgs/v1/200nodes/v1_report_tabbed.txt
@@ -0,0 +1,40 @@
+Experiment ID: 73d1ecf6-2dbe-4288-bd84-72e286d939be  Experiment ID: b3447e8e-f85e-48be-bb05-bf1b35b9e8b6  Experiment ID: 7b76c980-4882-4be7-898f-7ba7c4c950a2
+    Connections: 1                                       Connections: 2                                   Connections: 4
+    Rate: 200                                            Rate: 200                                        Rate: 200
+    Size: 1024                                           Size: 1024                                       Size: 1024
+    Total Valid Tx: 17800                                Total Valid Tx: 34600                            Total Valid Tx: 50464
+    Total Negative Latencies: 0                          Total Negative Latencies: 0                      Total Negative Latencies: 0
+    Minimum Latency: 1.710409731s                        Minimum Latency: 1.934148332s                    Minimum Latency: 3.207030208s
+    Maximum Latency: 8.977271598s                        Maximum Latency: 19.921012538s                   Maximum Latency: 22.695517951s
+    Average Latency: 3.873914787s                        Average Latency: 6.759146915s                    Average Latency: 9.394390517s
+    Standard Deviation: 1.80382447s                      Standard Deviation: 4.158131769s                 Standard Deviation: 4.907778924s
+Experiment ID: 240abfc5-2e9f-4096-8d25-87b8890b419f  Experiment ID: 99e88ddf-f0bd-4d44-98ac-cee52e2a74a6  Experiment ID: 88664d81-9d37-4820-a60e-8c0d2a9b2d63
+    Connections: 1                                       Connections: 2                                   Connections: 4
+    Rate: 400                                            Rate: 400                                        Rate: 400
+    Size: 1024                                           Size: 1024                                       Size: 1024
+    Total Valid Tx: 31200                                Total Valid Tx: 54706                            Total Valid Tx: 49463
+    Total Negative Latencies: 0                          Total Negative Latencies: 0                      Total Negative Latencies: 0
+    Minimum Latency: 2.050980308s                        Minimum Latency: 2.74995384s                     Minimum Latency: 4.469911766s
+    Maximum Latency: 26.195522089s                       Maximum Latency: 23.182542187s                   Maximum Latency: 16.480603178s
+    Average Latency: 9.280762294s                        Average Latency: 9.360818846s                    Average Latency: 9.976733037s
+    Standard Deviation: 7.166791513s                     Standard Deviation: 4.378492426s                 Standard Deviation: 3.223167612s
+Experiment ID: 3efdd2a0-8bef-43d0-a0f0-6dce2b63825f  Experiment ID: 3a49bc80-f63f-4d97-9663-3a02838fe1e8  Experiment ID: 1c2492ff-b82b-48a2-a975-bb252365521a
+    Connections: 1                                       Connections: 2                                   Connections: 4
+    Rate: 800                                            Rate: 800                                        Rate: 800
+    Size: 1024                                           Size: 1024                                       Size: 1024
+    Total Valid Tx: 51146                                Total Valid Tx: 51917                            Total Valid Tx: 41376
+    Total Negative Latencies: 0                          Total Negative Latencies: 0                      Total Negative Latencies: 0
+    Minimum Latency: 3.026020902s                        Minimum Latency: 4.617306731s                    Minimum Latency: 4.400844549s
+    Maximum Latency: 21.969169815s                       Maximum Latency: 29.138788503s                   Maximum Latency: 39.972301945s
+    Average Latency: 9.039266773s                        Average Latency: 11.676386139s                   Average Latency: 16.583749953s
+    Standard Deviation: 4.734959842s                     Standard Deviation: 6.190789791s                 Standard Deviation: 8.729665317s
+Experiment ID: 685cca77-ce3b-483f-9af8-6eafbdb03f7f  Experiment ID: ba5cd52a-e3e6-4e35-86e3-5b391395650b  Experiment ID: d27c4154-7c64-4379-91f9-0dc9b5f4c1d0
+    Connections: 1                                       Connections: 2                                   Connections: 4
+    Rate: 1600                                           Rate: 1600                                       Rate: 1600
+    Size: 1024                                           Size: 1024                                       Size: 1024
+    Total Valid Tx: 50889                                Total Valid Tx: 47732                            Total Valid Tx: 45530
+    Total Negative Latencies: 0                          Total Negative Latencies: 0                      Total Negative Latencies: 0
+    Minimum Latency: 4.411916099s                        Minimum Latency: 4.734922576s                    Minimum Latency: 5.500813279s
+    Maximum Latency: 24.512517023s                       Maximum Latency: 21.733104885s                   Maximum Latency: 30.120411703s
+    Average Latency: 11.022462182s                       Average Latency: 12.420967288s                   Average Latency: 13.648996358s
+    Standard Deviation: 4.882919113s                     Standard Deviation: 4.803231316s                 Standard Deviation: 6.370448765s
diff --git a/...des_with_latency_emulation/latencies/e_8190e83a-9135-444b-92fb-4efaeaaf2b52.png b/...des_with_latency_emulation/latencies/e_8190e83a-9135-444b-92fb-4efaeaaf2b52.png
diff --git a/docs/references/qa/imgs/v1/200nodes_with_latency_emulation/metrics/avg_cpu.png b/docs/references/qa/imgs/v1/200nodes_with_latency_emulation/metrics/avg_cpu.png
diff --git a/docs/references/qa/imgs/v1/200nodes_with_latency_emulation/metrics/avg_memory.png b/docs/references/qa/imgs/v1/200nodes_with_latency_emulation/metrics/avg_memory.png
diff --git a/...erences/qa/imgs/v1/200nodes_with_latency_emulation/metrics/avg_mempool_size.png b/...erences/qa/imgs/v1/200nodes_with_latency_emulation/metrics/avg_mempool_size.png
diff --git a/docs/references/qa/imgs/v1/200nodes_with_latency_emulation/metrics/block_rate.png b/docs/references/qa/imgs/v1/200nodes_with_latency_emulation/metrics/block_rate.png
diff --git a/...ences/qa/imgs/v1/200nodes_with_latency_emulation/metrics/block_rate_regular.png b/...ences/qa/imgs/v1/200nodes_with_latency_emulation/metrics/block_rate_regular.png
diff --git a/...erences/qa/imgs/v1/200nodes_with_latency_emulation/metrics/block_size_bytes.png b/...erences/qa/imgs/v1/200nodes_with_latency_emulation/metrics/block_size_bytes.png
diff --git a/docs/references/qa/imgs/v1/200nodes_with_latency_emulation/metrics/blocks.png b/docs/references/qa/imgs/v1/200nodes_with_latency_emulation/metrics/blocks.png
diff --git a/docs/references/qa/imgs/v1/200nodes_with_latency_emulation/metrics/cpu.png b/docs/references/qa/imgs/v1/200nodes_with_latency_emulation/metrics/cpu.png
diff --git a/docs/references/qa/imgs/v1/200nodes_with_latency_emulation/metrics/memory.png b/docs/references/qa/imgs/v1/200nodes_with_latency_emulation/metrics/memory.png
diff --git a/.../references/qa/imgs/v1/200nodes_with_latency_emulation/metrics/mempool_size.png b/.../references/qa/imgs/v1/200nodes_with_latency_emulation/metrics/mempool_size.png
diff --git a/...erences/qa/imgs/v1/200nodes_with_latency_emulation/metrics/mempool_size_max.png b/...erences/qa/imgs/v1/200nodes_with_latency_emulation/metrics/mempool_size_max.png
diff --git a/docs/references/qa/imgs/v1/200nodes_with_latency_emulation/metrics/peers.png b/docs/references/qa/imgs/v1/200nodes_with_latency_emulation/metrics/peers.png
diff --git a/docs/references/qa/imgs/v1/200nodes_with_latency_emulation/metrics/rounds.png b/docs/references/qa/imgs/v1/200nodes_with_latency_emulation/metrics/rounds.png
diff --git a/docs/references/qa/imgs/v1/200nodes_with_latency_emulation/metrics/total_txs.png b/docs/references/qa/imgs/v1/200nodes_with_latency_emulation/metrics/total_txs.png
diff --git a/...eferences/qa/imgs/v1/200nodes_with_latency_emulation/metrics/total_txs_rate.png b/...eferences/qa/imgs/v1/200nodes_with_latency_emulation/metrics/total_txs_rate.png
diff --git a/...s/qa/imgs/v1/200nodes_with_latency_emulation/metrics/total_txs_rate_regular.png b/...s/qa/imgs/v1/200nodes_with_latency_emulation/metrics/total_txs_rate_regular.png
diff --git a/docs/references/qa/imgs/v1/saturation/saturation-plotter-LE.py b/docs/references/qa/imgs/v1/saturation/saturation-plotter-LE.py
@@ -0,0 +1,54 @@
+#/usr/bin/env python3
+"""Plotter for comparing saturation results on v1 with and without Letency Emulation.
+
+This script generates an image with the number of processed transactions for different load
+configurations (tx rate and number of connections). The purpose is to find the saturation point of
+the network and to compare the results between different CometBFT versions.
+
+Quick setup before running:
+```
+python3 -m venv .venv && source .venv/bin/activate
+pip install matplotlib
+```
+"""
+import matplotlib.pyplot as plt
+import numpy as np
+
+# Transaction rate (x axis)
+rates = np.arange(100, 1100, 100)
+rates2 = [200, 400, 800, 1600]
+
+# expected values
+expected = [(i+1) * 100 * 89 for i in range(10)]
+
+# v1 without LE
+d1 = [8900,17800,26053,28800,32513,30455,33077,32191,30688,32395] # experiments/2024-03-26-13_47_51N/validator174
+d2 = [8900,17800,26300,25400,31371,31063,31603,32886,24521,25211] # experiments/2024-03-26-11_17_58N/validator174
+d3 = [8900,17800,26700,35600,38500,40502,51962,48328,50713,42361] # experiments/2024-03-26-20_20_33N/validator174
+
+# v1 with LE
+le1 = [8900,17800,26700,35600,34504,42169,38916,38004,34332,36948] # experiments/2024-03-25-17_41_09N/validator174
+le2 = [17800, 33800, 34644, 43464] # experiments/2024-03-25-12_17_12N/validator174
+le3 = [8900,17800,26700,33200,37665,51771,38631,49290,51526,46902] # experiments/2024-03-26-22_21_31N/validator174
+
+fig, ax = plt.subplots(figsize=(10, 5))
+ax.plot(rates, expected, linestyle='dotted', marker=',', color='g', label='expected')
+ax.plot(rates, d1, linestyle='solid', marker='o', color='b', label='without LE #1')
+ax.plot(rates, d2, linestyle='solid', marker='o', color='violet', label='without LE #2')
+ax.plot(rates, d3, linestyle='solid', marker='o', color='grey', label='without LE #3')
+ax.plot(rates, le1, linestyle='dashed', marker='s', color='r', label='with LE #1')
+ax.plot(rates2, le2, linestyle='dashed', marker='s', color='orange', label='with LE #2')
+ax.plot(rates, le3, linestyle='dashed', marker='s', color='brown', label='with LE #3')
+
+plt.title('saturation point for v1.0.0-alpha.2, c=1')
+plt.xlabel("rate (txs/s)")
+plt.ylabel("txs processed in 90 seconds")
+plt.xticks(np.arange(0, 1100, 200).tolist()) 
+ax.set_xlim([0, 1100])
+ax.set_ylim([0, 60000])
+ax.grid()
+ax.legend()
+
+fig.tight_layout()
+fig.savefig("saturation_v1_LE.png")
+plt.show()
diff --git a/docs/references/qa/imgs/v1/saturation/saturation-plotter.py b/docs/references/qa/imgs/v1/saturation/saturation-plotter.py
@@ -0,0 +1,56 @@
+#/usr/bin/env python3
+"""Plotter for comparing saturation results on v1 and v0.38.
+
+This script generates an image with the number of processed transactions for different load
+configurations (tx rate and number of connections). The purpose is to find the saturation point of
+the network and to compare the results between different CometBFT versions.
+
+Quick setup before running:
+```
+python3 -m venv .venv && source .venv/bin/activate
+pip install matplotlib
+```
+"""
+import matplotlib.pyplot as plt
+import numpy as np
+
+# Expected values of processed transactions for a given transaction rate.
+rates0 = [0, 3600]
+expected = [r * 89 for r in rates0]
+
+# Transaction rate (x axis)
+rates1 = [200, 400, 800, 1600]
+rates2 = [r*2 for r in rates1]
+rates4 = [r*2 for r in rates2]
+
+# v1 (without latency emulation), for number of connections c in [1,2,4]
+c1 = [17800,31200,51146,50889]
+c2 = [34600,54706,51917,47732]
+c4 = [50464,49463,41376,45530]
+
+# v0.38, for number of connections c in [1,2,4]
+d1 = [17800,35600,36831,40600]
+d2 = [33259,41565,38686,45034]
+d4 = [33259,41384,40816,39830]
+
+fig, ax = plt.subplots(figsize=(12, 5))
+ax.plot(rates0, expected, linestyle='dotted', marker=',', color='g', label='expected')
+ax.plot(rates1, c1, linestyle='solid', marker='s', color='red', label='v1 c=1')
+ax.plot(rates2, c2, linestyle='solid', marker='s', color='salmon', label='v1 c=2')
+ax.plot(rates4, c4, linestyle='solid', marker='s', color='orange', label='v1 c=4')
+ax.plot(rates1, d1, linestyle='dashed', marker='o', color='blue', label='v0.38 c=1')
+ax.plot(rates2, d2, linestyle='dashed', marker='o', color='violet', label='v0.38 c=2')
+ax.plot(rates4, d4, linestyle='dashed', marker='o', color='purple', label='v0.38 c=4')
+
+plt.title('finding the saturation point')
+plt.xlabel("total rate over all connections (txs/s)")
+plt.ylabel("txs processed in 90 seconds")
+plt.xticks(np.arange(0, 3600, 200).tolist()) 
+ax.set_xlim([0, 3600])
+ax.set_ylim([0, 60000])
+ax.grid()
+ax.legend()
+
+fig.tight_layout()
+fig.savefig("saturation_v1_v038.png")
+plt.show()
diff --git a/docs/references/qa/imgs/v1/saturation/saturation_v1_LE.png b/docs/references/qa/imgs/v1/saturation/saturation_v1_LE.png
diff --git a/docs/references/qa/imgs/v1/saturation/saturation_v1_v038.png b/docs/references/qa/imgs/v1/saturation/saturation_v1_v038.png
diff --git a/docs/references/qa/imgs/v1/saturation/throughput_vs_latencies_v1_with_LE.png b/docs/references/qa/imgs/v1/saturation/throughput_vs_latencies_v1_with_LE.png
diff --git a/docs/references/qa/imgs/v1/saturation/throughput_vs_latencies_v1_without_LE.png b/docs/references/qa/imgs/v1/saturation/throughput_vs_latencies_v1_without_LE.png
diff --git a/scripts/qa/reporting/prometheus_plotter.py b/scripts/qa/reporting/prometheus_plotter.py
@@ -1,9 +1,15 @@
 # pip install numpy pandas matplotlib requests 
 
+<<<<<<< HEAD
+=======
+# Requirements:
+# pip install requests matplotlib pandas prometheus-pandas
+import os
+import requests 
+>>>>>>> 53e612794 (docs(qa/v1): Report on saturation point and 200-nodes test (#2667))
 import sys
 import os
 
-import matplotlib as mpl
 import matplotlib.pyplot as plt
 import matplotlib.dates as md
 
@@ -95,10 +101,56 @@
     (( 'rate(' + fork + '_consensus_total_txs[20s])*60', ext_time_window[0], ext_time_window[1], '1s'), 'total_txs_rate',    dict(ylabel='TXs/min',           xlabel='time (s)', title='Rate of transaction processing', legend=False, figsize=(10,6), grid=True), True),
 ]
 
+<<<<<<< HEAD
 queriesRotating = [
     (( 'rate(' + fork + '_consensus_height[20s])*60',    time_window[0], time_window[1], '1s'), 'rotating_block_rate',    dict(ylabel='blocks/min',     xlabel='time', title='Rate of Block Creation',         legend=False, figsize=(10,6), grid=True), False),
     (( 'rate(' + fork + '_consensus_total_txs[20s])*60', time_window[0], time_window[1], '1s'), 'rotating_txs_rate',      dict(ylabel='TXs/min',        xlabel='time', title='Rate of Transaction processing', legend=False, figsize=(10,6), grid=True), False),
     (( fork + '_consensus_height{job=~"ephemeral.*"} or ' + fork + '_blocksync_latest_block_height{job=~"ephemeral.*"}',
+=======
+def usage():
+    print("Usage:")
+    print(f"\t{sys.argv[0]} release_name start_time window_size test_case")
+    print("where:")
+    print(f"- start_time is a UTF time in '%Y-%m-%dT%H:%M:%SZ' format")
+    print(f"- window size is in seconds")
+    print(f"- test_case is one of {TEST_CASES}")
+    print(f"Example: \t{sys.argv[0]} v1.0.0-alpha.2 2024-03-21T08:45:23Z 180 200_nodes")
+    exit(1)
+
+
+def queries_200_nodes(time_window, ext_time_window):
+    return [
+        (( 'cometbft_mempool_size',                           time_window[0], time_window[1], '1s'), 'mempool_size',              dict(ylabel='TXs',               xlabel='time (s)', title='Mempool Size',                   legend=False, figsize=(10,6), grid=True, ylim=(0, 5100), kind='area',stacked=True), False),
+        (( 'avg(cometbft_mempool_size)',                      time_window[0], time_window[1], '1s'), 'avg_mempool_size',          dict(ylabel='TXs',               xlabel='time (s)', title='Average Mempool Size',           legend=False, figsize=(10,6), grid=True, ylim=(0, 5100)), False),
+        (( 'max(cometbft_mempool_size)',                      time_window[0], time_window[1], '1s'), 'mempool_size_max',          dict(ylabel='TXs',               xlabel='time (s)', title='Maximum Mempool Size',           legend=False, figsize=(10,6), grid=True, ylim=(0, 5100)), False),
+        (( 'cometbft_p2p_peers',                              time_window[0], time_window[1], '1s'), 'peers',                     dict(ylabel='# Peers',           xlabel='time (s)', title='Peers',                          legend=False, figsize=(10,6), grid=True, ylim=(0, 150)), True),
+        #(( 'cometbft_consensus_height',                       time_window[0], time_window[1], '1s'), 'blocks_regular',            dict(ylabel='# Blocks',           xlabel='time (s)', title='Blocks in time',                 legend=False, figsize=(10,6), grid=True), False), 
+        (( 'cometbft_consensus_rounds',                       time_window[0], time_window[1], '1s'), 'rounds',                    dict(ylabel='# Rounds',          xlabel='time (s)', title='Rounds per block',               legend=False, figsize=(10,6), grid=True, ylim=(0, 4)), False),
+        (( 'rate(cometbft_consensus_height[20s])*60',         time_window[0], time_window[1], '1s'), 'block_rate_regular',        dict(ylabel='Blocks/min',        xlabel='time (s)', title='Rate of block creation',         legend=False, figsize=(10,6), grid=True, ylim=(0, 120)), True),
+        #(( 'avg(rate(cometbft_consensus_height[20s])*60)',    time_window[0], time_window[1], '1s'), 'block_rate_avg_reg',        dict(ylabel='Blocks/min',        xlabel='time (s)', title='Rate of block creation',         legend=False, figsize=(10,6), grid=True), False),
+        #(( 'cometbft_consensus_total_txs',                    time_window[0], time_window[1], '1s'), 'total_txs_regular',         dict(ylabel='# TXs',             xlabel='time (s)', title='Transactions in time',           legend=False, figsize=(10,6), grid=True), False),
+        (( 'rate(cometbft_consensus_total_txs[20s])*60',      time_window[0], time_window[1], '1s'), 'total_txs_rate_regular',    dict(ylabel='TXs/min',           xlabel='time (s)', title='Rate of transaction processing', legend=False, figsize=(10,6), grid=True, ylim=(0, 50000)), True),
+        #(( 'avg(rate(cometbft_consensus_total_txs[20s])*60)', time_window[0], time_window[1], '1s'), 'total_txs_rate_avg_reg',    dict(ylabel='TXs/min',           xlabel='time (s)', title='Rate of transaction processing', legend=False, figsize=(10,6), grid=True), False),
+        (( 'process_resident_memory_bytes',                   time_window[0], time_window[1], '1s'), 'memory',                    dict(ylabel='Memory (bytes)',    xlabel='time (s)', title='Memory usage',                   legend=False, figsize=(10,6), grid=True, ylim=(0, 2e9)), False),
+        (( 'avg(process_resident_memory_bytes)',              time_window[0], time_window[1], '1s'), 'avg_memory',                dict(ylabel='Memory (bytes)',    xlabel='time (s)', title='Average Memory usage',           legend=False, figsize=(10,6), grid=True, ylim=(0, 2e9)), False),
+        (( 'node_load1',                                      time_window[0], time_window[1], '1s'), 'cpu',                       dict(ylabel='Load',              xlabel='time (s)', title='Node load',                      legend=False, figsize=(10,6), grid=True, ylim=(0, 6)), False), 
+        (( 'avg(node_load1)',                                 time_window[0], time_window[1], '1s'), 'avg_cpu',                   dict(ylabel='Load',              xlabel='time (s)', title='Average Node load',              legend=False, figsize=(10,6), grid=True, ylim=(0, 6)), False),
+        (( 'cometbft_consensus_block_size_bytes/1024/1024',   time_window[0], time_window[1], '1s'), 'block_size_bytes',          dict(ylabel='Mb',                xlabel='time (s)', title='Block size (Mb)',                legend=False, figsize=(10,6), grid=True, ylim=(0, 4.1)), False),
+
+        # Extended window metrics
+        (( 'cometbft_consensus_height',                       ext_time_window[0], ext_time_window[1], '1s'), 'blocks',            dict(ylabel='# Blocks',          xlabel='time (s)', title='Blocks in time',                 legend=False, figsize=(10,6), grid=True), False), 
+        (( 'rate(cometbft_consensus_height[20s])*60',         ext_time_window[0], ext_time_window[1], '1s'), 'block_rate',        dict(ylabel='Blocks/min',        xlabel='time (s)', title='Rate of block creation',         legend=False, figsize=(10,6), grid=True), True),
+        (( 'cometbft_consensus_total_txs',                    ext_time_window[0], ext_time_window[1], '1s'), 'total_txs',         dict(ylabel='# TXs',             xlabel='time (s)', title='Transactions in time',           legend=False, figsize=(10,6), grid=True), False),
+        (( 'rate(cometbft_consensus_total_txs[20s])*60',      ext_time_window[0], ext_time_window[1], '1s'), 'total_txs_rate',    dict(ylabel='TXs/min',           xlabel='time (s)', title='Rate of transaction processing', legend=False, figsize=(10,6), grid=True, ylim=(0, 50000)), True),
+    ]
+
+
+def queries_rotating(time_window):
+    return [
+        (( 'rate(cometbft_consensus_height[20s])*60',    time_window[0], time_window[1], '1s'), 'rotating_block_rate',    dict(ylabel='blocks/min',     xlabel='time', title='Rate of Block Creation',         legend=False, figsize=(10,6), grid=True), False),
+        (( 'rate(cometbft_consensus_total_txs[20s])*60', time_window[0], time_window[1], '1s'), 'rotating_txs_rate',      dict(ylabel='TXs/min',        xlabel='time', title='Rate of Transaction processing', legend=False, figsize=(10,6), grid=True), False),
+        (( 'cometbft_consensus_height{job=~"ephemeral.*"} or cometbft_blocksync_latest_block_height{job=~"ephemeral.*"}',
+>>>>>>> 53e612794 (docs(qa/v1): Report on saturation point and 200-nodes test (#2667))
                                                          time_window[0], time_window[1], '1s'), 'rotating_eph_heights',   dict(ylabel='height',         xlabel='time', title='Heights of Ephemeral Nodes',     legend=False, figsize=(10,6), grid=True), False),
     (( fork + '_p2p_peers',                              time_window[0], time_window[1], '1s'), 'rotating_peers',         dict(ylabel='# peers',        xlabel='time', title='Peers',                          legend=False, figsize=(10,6), grid=True), False),
     (( 'avg(process_resident_memory_bytes)',             time_window[0], time_window[1], '1s'), 'rotating_avg_memory',    dict(ylabel='memory (bytes)', xlabel='time', title='Average Memory Usage',           legend=False, figsize=(10,6), grid=True), False),