ENH: ndimage.uniform_filter: implement nan_policy
#17393
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 3 commits
November 10, 2022 12:43
Have created a new C-level function that checks/ignores NaN values when computing boxcar averages for the uniform_filter function. This new function is almost identical to the original, but now checks/ignores NaN values when summing and adjusts the number of values in the average based on finite values; i.e., a numpy.nanmean()-like average is taken over the window. At the python level, an 'ignore_nan' keyword has been added to both the uniform_filter and uniform_filter1d functions. By default, the original behavior is maintained (i.e., ignore_nan=False) and no performance penalty is incurred on existing code. Tests have been added to the testing suite to check that NaN ignoring is working correctly. A new benchmark has been added to the benchmarking suite; tests indicate very minimal impacts to performance when using ignore_nan=True as opposted to ignore_nan=False. More robust benchmarking should be performed to fully understand any preformance impacts; however, getting the 'right' answer (i.e., not propogating NaNs through entire dataset) is typically more important than being blazing fast. See scipy#7818
Earlier commit implemented ignoring of NaNs in the uniform_filter and uniform_filter1d functions. This commit moves to using the nan_policy keyword argument for determining how NaNs are to be handled so that API calls are consistent across scipy. Closes scipy#7818
|
Ooo! I somehow only just noticed this. I haven't tested it, but having this would be lovely. |
lucascolley
changed the title
nan_policy for uniform_filter
lucascolley
changed the title
nan_policy for uniform_filterndimage.uniform_filter: implement nan_policy
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Earlier commit (1871349) implemented ignoring of NaNs in the uniform_filter and uniform_filter1d functions. This commit moves to using the nan_policy keyword argument for determining how NaNs are to be handled so that API calls are consistent across scipy.
Have created a new C-level function that checks/ignores NaN values when computing boxcar averages for the uniform_filter function. This new function is almost identical to the original, but now checks/ignores NaN values when summing and adjusts the number of values in the average based on finite values; i.e., a numpy.nanmean()-like average is taken over the window.
Tests have been added to the testing suite to check that NaN ignoring is working correctly.
A new benchmark has been added to the benchmarking suite; tests indicate very minimal impacts to performance when using ignore_nan=True as opposted to ignore_nan=False. More robust benchmarking should be performed to fully understand any preformance impacts; however, getting the 'right' answer (i.e., not propogating NaNs through entire dataset) is typically more important than being blazing fast.
Closes #7818
Reference issue
What does this implement/fix?
Additional information