Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmarking suite #2188

Closed
mdboom opened this issue Jul 1, 2013 · 13 comments
Closed

Add benchmarking suite #2188

mdboom opened this issue Jul 1, 2013 · 13 comments
Assignees
Labels
Difficulty: Medium https://matplotlib.org/devdocs/devel/contribute.html#good-first-issues Good first issue Open a pull request against these issues if there are no active ones! New feature Performance status: closed as inactive Issues closed by the "Stale" Github Action. Please comment on any you think should still be open. status: inactive Marked by the “Stale” Github Action

Comments

@mdboom
Copy link
Member

mdboom commented Jul 1, 2013

It would be nice to have a standard suite of benchmarks -- particularly those things that are known performance bottlenecks -- so we can track their improvement or regression over time and provide data about the relative performance of different backends etc.

@tacaswell tacaswell added this to the v1.5.x milestone Aug 17, 2014
@tacaswell
Copy link
Member

@mdboom seems like you already solved this https://github.com/spacetelescope/asv

Can we assign this to your mac-mini?

@petehuang
Copy link
Contributor

Hi,

This issue has been inactive for a while. Are there any updates? Any initial ideas that we have to help people start thinking about this? Thanks!

@tacaswell tacaswell added new-contributor-friendly Difficulty: Medium https://matplotlib.org/devdocs/devel/contribute.html#good-first-issues labels Dec 29, 2016
@tacaswell
Copy link
Member

To increase visibility tagged as 'new-contributor-friendly' as this is a very self-contained project, but medium difficulty as it will involve sorting out how to set up ASV and how to find reliable hardware to run it on (our typical CI tricks won't work here as we need fixed resources underneath). Maybe pricing AWS for this?

@dopplershift
Copy link
Contributor

Rackspace may also be an option; I think that's what powers https://speed.python.org/

@tacaswell tacaswell modified the milestones: 2.1 (next point release), 2.2 (next next feature release) Oct 3, 2017
@tacaswell tacaswell added the Good first issue Open a pull request against these issues if there are no active ones! label Oct 16, 2017
@jklymak
Copy link
Member

jklymak commented Jan 25, 2018

Having the suite alone would be a useful first step. Automating it could come later.

@QuLogic
Copy link
Member

QuLogic commented Jan 25, 2018

I am working on this; will try to publish something this weekend.

@QuLogic QuLogic self-assigned this Jan 25, 2018
@jklymak
Copy link
Member

jklymak commented Jan 25, 2018

I'll bump the milestone though, as I don't think there is a higher rush. Just getting into master should be enough.

@jklymak jklymak modified the milestones: v2.2, v3.0 Jan 25, 2018
@QuLogic
Copy link
Member

QuLogic commented Feb 4, 2018

We have some benchmarks here with results that end up here, from about 1.5.0 onwards. Those results are produced on my machine and I try to run them while I'm not using it, but there's not real guarantee of independence there.

We are working to get it running on the same dedicated hardware used for Pandas benchmarking.

@anntzer
Copy link
Contributor

anntzer commented Feb 18, 2018

Perhaps worth adding a benchmark on just the time to import matplotlib, or matplotlib.pyplot (but forcing the backend to agg, as that's going to be backend dependent).

@tacaswell
Copy link
Member

import / start up time has been something that @efiring is concerned about.

@QuLogic
Copy link
Member

QuLogic commented Feb 20, 2018

There's a PR for that in asv: airspeed-velocity/asv#487 but it seems to have stalled.

@QuLogic
Copy link
Member

QuLogic commented Oct 17, 2018

Waiting on asv-runner/asv-runner#2, though I think I need to update to asv 0.3 as well.

@github-actions
Copy link

github-actions bot commented Mar 2, 2023

This issue has been marked "inactive" because it has been 365 days since the last comment. If this issue is still present in recent Matplotlib releases, or the feature request is still wanted, please leave a comment and this label will be removed. If there are no updates in another 30 days, this issue will be automatically closed, but you are free to re-open or create a new issue if needed. We value issue reports, and this procedure is meant to help us resurface and prioritize issues that have not been addressed yet, not make them disappear. Thanks for your help!

@github-actions github-actions bot added the status: inactive Marked by the “Stale” Github Action label Mar 2, 2023
@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Apr 2, 2023
@rcomer rcomer added the status: closed as inactive Issues closed by the "Stale" Github Action. Please comment on any you think should still be open. label May 30, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Difficulty: Medium https://matplotlib.org/devdocs/devel/contribute.html#good-first-issues Good first issue Open a pull request against these issues if there are no active ones! New feature Performance status: closed as inactive Issues closed by the "Stale" Github Action. Please comment on any you think should still be open. status: inactive Marked by the “Stale” Github Action
Projects
None yet
Development

No branches or pull requests

9 participants