Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase number of runs for all performance benchmarks #1315

Closed
sequba opened this issue Sep 26, 2023 · 1 comment · Fixed by #1366
Closed

Increase number of runs for all performance benchmarks #1315

sequba opened this issue Sep 26, 2023 · 1 comment · Fixed by #1366
Assignees
Labels
Impact: Medium Performance Issues related to the performance Tech Debt Tests Related to tests or testing environment

Comments

@sequba
Copy link
Contributor

sequba commented Sep 26, 2023

After increasing the number of runs for basic performance benchmarks (currently 100 runs, #1303), I see much less noise in their results (see screenshot). The benchmarks will be much more useful to us if they were run less often but with greater accuracy (I'd make all benchmarks run 100 times).

Example performance report: first 4 benchmarks run 100 times, last 5 run only once:
image

@sequba sequba self-assigned this Sep 26, 2023
@sequba sequba added Tests Related to tests or testing environment Performance Issues related to the performance Impact: Medium labels Sep 26, 2023
@sequba sequba linked a pull request Jan 10, 2024 that will close this issue
14 tasks
@sequba
Copy link
Contributor Author

sequba commented Feb 15, 2024

Released with version 2.6.2.

@sequba sequba closed this as completed Feb 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Impact: Medium Performance Issues related to the performance Tech Debt Tests Related to tests or testing environment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant