Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: add codspeed benchmarks #37

Closed
wants to merge 11 commits into from
Closed

Conversation

tlambert03
Copy link
Member

@tlambert03 tlambert03 commented Mar 28, 2023

this was accidentally committed this to main and then reverted... testing benchmarks on CI with codspeed.io

@codecov-commenter
Copy link

codecov-commenter commented Mar 28, 2023

Codecov Report

Merging #37 (794c04e) into main (5f42466) will not change coverage.
The diff coverage is n/a.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@           Coverage Diff           @@
##             main      #37   +/-   ##
=======================================
  Coverage   92.96%   92.96%           
=======================================
  Files          29       29           
  Lines         725      725           
=======================================
  Hits          674      674           
  Misses         51       51           

@tlambert03
Copy link
Member Author

ok, some things learned:

  • the runner is much slower than my local computer. benchmarks take forever. So while codspeed.io probably would be good for catching regressions on critical functions run on smaller bits of data, it's probably not a good solution for measuring larger real-world datasets (that's harder of course, and likely scales with performance on smaller problems anyway)
  • the last benchmark job (with dramatically reduced data size) did run successfully, but failed on upload: https://github.com/funkelab/motile/actions/runs/4544180511/jobs/8009860945?pr=37 ... so would require @funkey or someone with admin rights to the funkelab organization to accept the request from codspeed that I recently sent.

will probably pursue different strategies (like line profiling which would be more immediately actionable anyway) and let this sit here until further discussion

@tlambert03 tlambert03 marked this pull request as draft November 2, 2023 16:11
@tlambert03 tlambert03 closed this Nov 6, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants