You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We want to design a simple stress test tool for the aggregator and implement it in its first version.
The idea is to be able to verify that under a load of ~100 simulated signers, the aggregator:
Handles all Signer Key Registrations without error in a reasonable time (<30s per request)
Handles all Signer Signatures Registrations without error in a reasonable time (<30s per request)
Creates the associated certificates and artifacts for each signed entity type (<300s to retrieve the information)
All of these expectations are under a load of ~10-100 simulated clients requesting the certificates and artifacts routes, retrieving responses in a reasonable time (<10s per request)
Note: The simulated signers and clients should be represented by the HTTP calls they do on the aggregator REST API
Also, we want to understand and keep track of the limitations of this design which is expected to run on a single computer, and that will be enhanced in future issues. We will probably add retry strategies, ...
Nominal scenario
Phase 1: Run Without Clients
1. Create the Signer Fixtures
2. Launch aggregator
3. Wait for the aggregator REST API to be up (/epoch-settings route is answering 200)
4. Create and send the Signer Key Registrations payloads
5. Move the aggregator 1 epoch forward and wait for the aggregator to be at new epoch (/epoch-settings route)
6. Create and send the Signer Key Registrations payloads
7. Compute genesis certificate (no need to stop the aggregator)
8. Move the aggregator 1 epoch forward and wait for the aggregator to be at new epoch (/epoch-settings route)
9. Create and send the Signer Key Registrations payloads
10. Wait for pending certificate to be available for Mithril Stake Distribution (/certificate-pending route is answering 200)
11. Create the Signer Signatures Registrations payloads
12. Send the Signer Signatures Registrations payloads
13. Wait for the certificate to be available for Mithril Stake Distribution (/certificates route is answering 200)
14. Wait for the artifact to be available for Mithril Stake Distribution (/artifact/mithril-stake-distributions route is answering 200)
15. Wait for pending certificate to be available for Cardano Immutable Files (/certificate-pending route is answering 200)
16. Create the Signer Signatures Registrations payloads
17. Send the Signer Signatures Registrations payloads
18. Wait for the certificate to be available for Cardano Immutable Files (/certificates route is answering 200)
19. Wait for the artifact to be available for Cardano Immutable Files (/artifact/snapshots route is answering 200)
Phase 2: Run With Clients
Create the Clients payloads
Send in a separate thread the Clients payload with pseudo-random pace
Move the aggregator 1 epoch forward
Create and send the Signer Key Registrations payloads
Start same as Phase 1 step 4
To do
Compute statistics for HTTP Client requests (min/max, avg, std, percentiles, ...) (e.g. with rewrk crate)
Compute summary statistics for the whole test
Measure performance of aggregator process and compute statistics
Other tasks
Update the README with command to run the stress test
Definition of Success/Failure
The test fails if:
Any step fails with a timeout
The aggregator crashes
The test succeeds otherwise
We will display statistics for each eligible step (error rates, error distributions, min/max, ...) and for clients requests (To be defined)
During single signatures registrations, the performance are degraded as the multi-signer is re-computed constantly (and the time to create it is quadratic with the number of registered signers)
jpraynaud
changed the title
Design & implement basic stress test tool for aggregator
Design & implement stress test tool for aggregator
Jul 31, 2023
jpraynaud
changed the title
Design & implement stress test tool for aggregator
Design & implement stress test tool for aggregator - phase 1
Aug 10, 2023
Issue
We want to design a simple stress test tool for the aggregator and implement it in its first version.
The idea is to be able to verify that under a load of
~100
simulated signers, the aggregator:<30s
per request)<30s
per request)<300s
to retrieve the information)~10-100
simulated clients requesting the certificates and artifacts routes, retrieving responses in a reasonable time (<10s
per request)Note: The simulated signers and clients should be represented by the HTTP calls they do on the aggregator REST API
Also, we want to understand and keep track of the limitations of this design which is expected to run on a single computer, and that will be enhanced in future issues. We will probably add retry strategies, ...
Nominal scenario
Phase 1: Run Without Clients
1. Create the Signer Fixtures
2. Launch aggregator
3. Wait for the aggregator REST API to be up (
/epoch-settings
route is answering200
)4. Create and send the Signer Key Registrations payloads
5. Move the aggregator
1
epoch forward and wait for the aggregator to be at new epoch (/epoch-settings
route)6. Create and send the Signer Key Registrations payloads
7. Compute genesis certificate (no need to stop the aggregator)
8. Move the aggregator
1
epoch forward and wait for the aggregator to be at new epoch (/epoch-settings
route)9. Create and send the Signer Key Registrations payloads
10. Wait for pending certificate to be available for Mithril Stake Distribution (
/certificate-pending
route is answering200
)11. Create the Signer Signatures Registrations payloads
12. Send the Signer Signatures Registrations payloads
13. Wait for the certificate to be available for Mithril Stake Distribution (
/certificates
route is answering200
)14. Wait for the artifact to be available for Mithril Stake Distribution (
/artifact/mithril-stake-distributions
route is answering200
)15. Wait for pending certificate to be available for Cardano Immutable Files (
/certificate-pending
route is answering200
)16. Create the Signer Signatures Registrations payloads
17. Send the Signer Signatures Registrations payloads
18. Wait for the certificate to be available for Cardano Immutable Files (
/certificates
route is answering200
)19. Wait for the artifact to be available for Cardano Immutable Files (
/artifact/snapshots
route is answering200
)Phase 2: Run With Clients
1
epoch forwardTo do
rewrk
crate)Other tasks
README
with command to run the stress testDefinition of Success/Failure
We will display statistics for each eligible step (error rates, error distributions, min/max, ...) and for clients requests (To be defined)
Design
Here is the design that we have created:
Parent issue
#904
The text was updated successfully, but these errors were encountered: