-
-
Notifications
You must be signed in to change notification settings - Fork 7.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add benchmarking suite #2188
Comments
@mdboom seems like you already solved this https://github.com/spacetelescope/asv Can we assign this to your mac-mini? |
Hi, This issue has been inactive for a while. Are there any updates? Any initial ideas that we have to help people start thinking about this? Thanks! |
To increase visibility tagged as 'new-contributor-friendly' as this is a very self-contained project, but medium difficulty as it will involve sorting out how to set up ASV and how to find reliable hardware to run it on (our typical CI tricks won't work here as we need fixed resources underneath). Maybe pricing AWS for this? |
Rackspace may also be an option; I think that's what powers https://speed.python.org/ |
Having the suite alone would be a useful first step. Automating it could come later. |
I am working on this; will try to publish something this weekend. |
I'll bump the milestone though, as I don't think there is a higher rush. Just getting into master should be enough. |
We have some benchmarks here with results that end up here, from about 1.5.0 onwards. Those results are produced on my machine and I try to run them while I'm not using it, but there's not real guarantee of independence there. We are working to get it running on the same dedicated hardware used for Pandas benchmarking. |
Perhaps worth adding a benchmark on just the time to import matplotlib, or matplotlib.pyplot (but forcing the backend to agg, as that's going to be backend dependent). |
import / start up time has been something that @efiring is concerned about. |
There's a PR for that in asv: airspeed-velocity/asv#487 but it seems to have stalled. |
Waiting on asv-runner/asv-runner#2, though I think I need to update to asv 0.3 as well. |
This issue has been marked "inactive" because it has been 365 days since the last comment. If this issue is still present in recent Matplotlib releases, or the feature request is still wanted, please leave a comment and this label will be removed. If there are no updates in another 30 days, this issue will be automatically closed, but you are free to re-open or create a new issue if needed. We value issue reports, and this procedure is meant to help us resurface and prioritize issues that have not been addressed yet, not make them disappear. Thanks for your help! |
It would be nice to have a standard suite of benchmarks -- particularly those things that are known performance bottlenecks -- so we can track their improvement or regression over time and provide data about the relative performance of different backends etc.
The text was updated successfully, but these errors were encountered: