Add CodSpeed continuous benchmarking#772
Closed
codspeed-hq[bot] wants to merge 1 commit into
Closed
Conversation
Author
Collaborator
|
unneeded as already covered by #771 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Continuous performance testing with CodSpeed
This PR wires up CodSpeed so linopy's performance is measured automatically on every pull request, catching regressions before they reach
master.linopy already ships a well-structured
pytest-benchmarksuite underbenchmarks/. CodSpeed integrates with that suite directly throughpytest-codspeed, which provides a drop-inbenchmarkfixture — so no benchmark code needed to change.What's included
.github/workflows/codspeed.yml— a new workflow that builds the package, installspytest-codspeed, and runs the benchmarks under CodSpeed's CPU simulation instrument on pull requests, pushes tomaster, and manual dispatch. It authenticates via OpenID Connect (no token to manage) and runs onubuntu-latest.Benchmarks measured
The workflow runs the deterministic, dependency-free models across all three phases in
--quickmode (the same quick mode documented inbenchmarks/README.md):basic,knapsack,expression_arithmetic,sparse_networktest_build.py), LP write (test_lp_write.py), and matrix generation (test_matrices.py)CPU simulation mode is used because these benchmarks are CPU-bound pure-Python/numpy code, where simulation gives hardware-agnostic, low-variance measurements with automatic flame graphs.
Notes and next steps
test_pypsa_carbon_management.pyand thepypsa_scigridparametrizations require downloading external example networks at runtime, which makes them unsuitable for deterministic CI measurement. They remain available for local runs and can be added later (for example on CodSpeed Macro Runners with walltime mode) if desired.codspeed run --mode simulation, with results linking to thePyPSA/linopyproject.workflow_dispatch(or let the next PR run it) so CodSpeed can establish the performance baseline. The badge and PR performance reports populate automatically from then on.