Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Fixes for asv benchmarks #1576

Closed
wants to merge 1 commit into from
Closed

Conversation

luizirber
Copy link
Member

In #1571 @mr-eyes mentioned he couldn't run the asv benchmarks. I also had problems fixing them for #1564. It seems that one of the main issues is that virtualenv was using a very old version (because asv had issues with newer versions), but it seemed to be fixed and it is working with newer virtualenv again).

This PR avoid pinning the virtualenv version, and is enough to make it work in my system.

@mr-eyes, can you try this branch and see if benchmarks (with tox -e asv) work?

@codecov
Copy link

codecov bot commented Jun 8, 2021

Codecov Report

Merging #1576 (828e89f) into latest (6b5806c) will increase coverage by 8.12%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           latest    #1576      +/-   ##
==========================================
+ Coverage   80.98%   89.11%   +8.12%     
==========================================
  Files         102       75      -27     
  Lines       10331     6623    -3708     
  Branches     1171     1171              
==========================================
- Hits         8367     5902    -2465     
+ Misses       1758      515    -1243     
  Partials      206      206              
Flag Coverage Δ
python 89.11% <ø> (ø)
rust ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/core/src/ffi/signature.rs
src/core/src/sketch/nodegraph.rs
src/core/src/ffi/minhash.rs
src/core/src/signature.rs
src/core/src/from.rs
src/core/src/ffi/nodegraph.rs
src/core/tests/test.rs
src/core/src/wasm.rs
src/core/src/index/sbt/mod.rs
src/core/src/index/storage.rs
... and 17 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6b5806c...828e89f. Read the comment docs.

@mr-eyes
Copy link
Member

mr-eyes commented Jun 9, 2021

Thanks, Luiz. I want to clarify the issue that I was experiencing.

When I tried tox -e asv on latest it was working fine. But when I add new functions to benchmarks.py I always get an error stating that benchmarking is getting slow due to the new functions.

Another error I got is that benchmarking is always set to run on Python 3.7, and actually, the sourmash_dev environment does not contain that version of Python. So I changed the asv.conf.json: Pythons to 3.9.

Update: Sometimes I had to remove the .tox directory to get it working.

@mr-eyes
Copy link
Member

mr-eyes commented Jun 9, 2021

I would like also to suggest adding instructions for the benchmarking in the Sourmash dev docs section.
Including but not limited to:

  • What is benchmarks.py and how it's working?
  • How to add new functions to benchmark?

@luizirber
Copy link
Member Author

I think this was fixed in #1834

@luizirber luizirber closed this Mar 27, 2022
@luizirber luizirber deleted the asv_benchmarks_fixes branch March 27, 2022 22:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants