-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a continuous benchmarking workflow #184
Comments
Yes, I'd recommend it. A problem with EpiNow2 is that the benchmarks take quite long to run (1-5 minutes) and so doing repeated testing with multiple tests inflates run times pretty quickly. Because of this we have small sample sizes for the repeated tests (n=5) and get quite lot of uncertainty in the benchmarks. I think it would work really well for less computation intensive code such as in styler. Perhaps worth noting that the package is looking for a maintainer |
Thanks @sbfnk - good to know - I'll look into adding this to the open PR. |
Just looking into this and something I should have realised earlier is that it won't be useful to add this workflow in the current PR as the function names are changing as well. I could add the old names as aliases to the functions, but that would have to be reversed eventually. Possibly best to add this workflow after the current PR is merged. |
This PR requests the addition of a GHA workflow for continuous benchmarking. This is expected to help avoid changes that reduce model run speed, which is important as the expected use case is 100s-1000s of runs.
An example can be found in {EpiNow2}. @sbfnk and @jamesmbaazam any feedback on your experience with using {touchstone} would be great - would you recommend using it?
The text was updated successfully, but these errors were encountered: