New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Accelerate import - prevent numba warnings #1378
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
github-actions
bot
added
indicators
Climate indices and indicators
sdba
Issues concerning the sdba submodule.
labels
May 26, 2023
Wow! |
Zeitsperre
approved these changes
May 26, 2023
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Would it be possible to deal with some of the numba-related warnings being raised right now? For instance:
|
Of course! I forgot to look into this module (cffwis) as it had no pre-compiled numba funcs. |
5 tasks
aulemahal
added a commit
that referenced
this pull request
Jun 8, 2023
<!--Please ensure the PR fulfills the following requirements! --> <!-- If this is your first PR, make sure to add your details to the AUTHORS.rst! --> ### Pull Request Checklist: - [ ] This PR addresses an already opened issue (for bug fixes / features) - This PR fixes #xyz - [ ] Tests for the changes have been added (for bug fixes / features) - [ ] (If applicable) Documentation has been added / updated (for bug fixes / features) - [ ] CHANGES.rst has been updated (with summary of main changes) - [ ] Link to issue (:issue:`number`) and pull request (:pull:`number`) has been added ### What kind of change does this PR introduce? PR #1378 introduced caching for the `guvectorize` functions. Turns out that this caching interferes with the development of these function as the cached version doesn't get updated when the code is modified. (As tested by @coxipi ) However, when installing a new version of xclim, cached copies are removed and thus updated. The bug mentionned above only affects locally installed versions (`pip install -e`). A note was added to the contributing notes. ### Does this PR introduce a breaking change? No ### Other information: No
5 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
approved
Approved for additional tests
indicators
Climate indices and indicators
sdba
Issues concerning the sdba submodule.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Pull Request Checklist:
xscen
developers.number
) and pull request (:pull:number
) has been addedWhat kind of change does this PR introduce?
PR #1135 removed all signatures where possible, so that
@jit
-decorated function would be compiled at run time (you know "just in time"...). However,@guvectorize
-decorated functions must have signatures and thus must be compiled at compile time, which means upon import.This PR adds
cache=True
to these function, so numba compiles them once and than saves the compilation to a__pycache__
folder if possible. The first import will be slow as before, but subsequent imports should be much faster.I'm sure there are edgecases where the user of xclim doesn't have write access to the source folder, which might mean this caching will be unavailable. But for most users, this should be a nice performance boost.
On my machine I went from 13s to 2.5s.
This PR also forces
nopython=True
to these@guvectorize
functions. Numba 0.59 will change the default value of this parameter and it was raising warning complaining about our lack of precision.Does this PR introduce a breaking change?
It shouldn't. But I do not know all the ramifications of caching these functions...
Other information:
@RondeauG, fyi.