Fix sorting and byte-order errors with numpy 1.25 #1670
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@mcflugen Wow, these are some nasty little bugs. Thank you for fixing them. |
|
🐛 🔥 |
|
It appears the failures in the notebook tests are a result of a new version of matplotlib (v3.7.2). If I limit matplotlib to less than 3.7.2, which I've done in this pull request, the failures go away. Note that the notebook tests only fail when they are run through the CI (I assume it has something to do with them being run on a headless server). I'll try to fix this matplotlib issue as a separate pull request. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This pull request fixes a set of errors that cropped up with a recent version of numpy (v1.25) and (maybe) netcdf.
There were two sets of errors. The first were some doctests that read a sample netcdf file that was, apparently, written as big-endian. With the new package versions, the arrays were read from the file with their byte order retained—regardless of the native byte order of the machine the tests were run on. To fix this, I've just modified the doctests to convert the arrays to native byteorder.
The second set of errors were due to a change in argsort on some architectures. Certain architectures now use a vectorized version of the quicksort algorithm that does does not guarantee deterministic results with argsort (of course the values are still sorted but, in the case of a tie, the indices are not always guaranteed to be in the one order). I've fixed this by using the
kind="stable"keyword for argsort.I've also fixed another related issue where a doctest (
FlowAccumulator.link_order_upstream) produced an array whose ordering was also not deterministic (possibly due to the underlying sorting algorithm of the architecture). The new tests check that downstream links always appear before upstream links but the specific ordering of links at an equal level is not guaranteed.Checklist - did you ...