Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simulator network configuration and stress test #4677

Merged
merged 8 commits into from
Sep 23, 2022
Merged

Conversation

mystenmark
Copy link
Contributor

@mystenmark mystenmark commented Sep 16, 2022

Support for configuring the network using the simulator's SimConfig struct. (See MystenLabs/mysten-sim#3 for background).

Adds a new test which runs the stress tester inside the simulator. This test can currently expose various bugs/errors when run with a higher latency network. The default target that is checked in uses low latency and thus will pass in CI.

@sadhansood is there an easy way to get a failure signal from the BenchDriver, e.g. a callback every time a transaction fails? Right now I don't have a good way to make this test fail if errors are encountered.

@mystenmark mystenmark force-pushed the mlogan-sim-config branch 3 times, most recently from 1747908 to 3c8ba2f Compare September 19, 2022 22:35
@mystenmark mystenmark marked this pull request as ready for review September 19, 2022 22:36
@mystenmark mystenmark changed the title Simulator configuration Simulator network configuration and stress test Sep 19, 2022
@sadhansood
Copy link
Contributor

@mystenmark the BenchmarkStats object that is returned by the BenchDriver has num_errors and num_success metric in it. We could pass in the run_duration as some finite value (5 mins, 1 hour, etc) and check the resulting stats for %error_rate and fail the test if it's too high.

@mystenmark
Copy link
Contributor Author

@mystenmark the BenchmarkStats object that is returned by the BenchDriver has num_errors and num_success metric in it. We could pass in the run_duration as some finite value (5 mins, 1 hour, etc) and check the resulting stats for %error_rate and fail the test if it's too high.

I took a look at that, but apparently the errors that I'm seeing aren't resulting in client facing errors, so they don't show up in the BenchmarkStats - I'll need to find some other way to instrument those.

I added a simple assert for 0 errors for now.

We can add longer tests, but I'll need to set up a nightly job for longer tests - I don't want to add very long running tests to CI.

@mystenmark mystenmark merged commit bbaafa2 into main Sep 23, 2022
@mystenmark mystenmark deleted the mlogan-sim-config branch September 23, 2022 19:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants