Config and proof-of-concept benchmarks with airspeed #443
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Airspeed (https://asv.readthedocs.io/en/stable/) is used by SciPy, NumPy etc to track the performance of each commit and catch perf regressions. They host their results online (https://pv.github.io/numpy-bench/) and run as part of the CI process but out of the box airspeed lets you serve a local html file.
This PR contains config and a basic set of benchmarks which, when run on the last 20 dev commits, generate the following:

(This is mostly for demo/discussion purposes, I could modify lots of our existing pytest suite to get more useful tracking and also look at remote hosting for the HTML if this is something we think would be useful.)
Types of changes
Checklist: