Skip to content

Add CodSpeed continuous benchmarking#772

Closed
codspeed-hq[bot] wants to merge 1 commit into
masterfrom
codspeed/wizard-1780913689368
Closed

Add CodSpeed continuous benchmarking#772
codspeed-hq[bot] wants to merge 1 commit into
masterfrom
codspeed/wizard-1780913689368

Conversation

@codspeed-hq
Copy link
Copy Markdown

@codspeed-hq codspeed-hq Bot commented Jun 8, 2026

Continuous performance testing with CodSpeed

This PR wires up CodSpeed so linopy's performance is measured automatically on every pull request, catching regressions before they reach master.

linopy already ships a well-structured pytest-benchmark suite under benchmarks/. CodSpeed integrates with that suite directly through pytest-codspeed, which provides a drop-in benchmark fixture — so no benchmark code needed to change.

What's included

  • .github/workflows/codspeed.yml — a new workflow that builds the package, installs pytest-codspeed, and runs the benchmarks under CodSpeed's CPU simulation instrument on pull requests, pushes to master, and manual dispatch. It authenticates via OpenID Connect (no token to manage) and runs on ubuntu-latest.
  • README badge — a CodSpeed badge next to the existing status badges.

Benchmarks measured

The workflow runs the deterministic, dependency-free models across all three phases in --quick mode (the same quick mode documented in benchmarks/README.md):

  • Models: basic, knapsack, expression_arithmetic, sparse_network
  • Phases: build (test_build.py), LP write (test_lp_write.py), and matrix generation (test_matrices.py)

CPU simulation mode is used because these benchmarks are CPU-bound pure-Python/numpy code, where simulation gives hardware-agnostic, low-variance measurements with automatic flame graphs.

Notes and next steps

  • PyPSA benchmarks are intentionally excluded. test_pypsa_carbon_management.py and the pypsa_scigrid parametrizations require downloading external example networks at runtime, which makes them unsuitable for deterministic CI measurement. They remain available for local runs and can be added later (for example on CodSpeed Macro Runners with walltime mode) if desired.
  • The benchmarks ran successfully locally under codspeed run --mode simulation, with results linking to the PyPSA/linopy project.
  • After this PR is merged, trigger the workflow once via workflow_dispatch (or let the next PR run it) so CodSpeed can establish the performance baseline. The badge and PR performance reports populate automatically from then on.

@codspeed-hq
Copy link
Copy Markdown
Author

codspeed-hq Bot commented Jun 8, 2026

Congrats! CodSpeed is installed 🎉

🆕 33 new benchmarks were detected.

You will start to see performance impacts in the reports once the benchmarks are run from your default branch.

Detected benchmarks


ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.


Open in CodSpeed

@codspeed-hq codspeed-hq Bot marked this pull request as ready for review June 8, 2026 10:30
@FabianHofmann
Copy link
Copy Markdown
Collaborator

unneeded as already covered by #771

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant