Skip to content

Commit

Permalink
allow numpy 1.24 (#770)
Browse files Browse the repository at this point in the history
Revert #759 and fix unit tests.

PR #759 excluded the latest Numpy version (1.24) because of failing unit
tests check:sqrt and check:log2-complex in Github actions due to lack of
accuracy. Confusingly, tests never failed locally, and only randomly on
Github - rerunning a failed job would occasionally result in success. Out
of caution, the version was excluded until the cause of this situation was
known.

The problem is now understood to be caused by vectorized routines that
make use of the AVX-512 CPU extensions. None of the local machines used
in testing supported these instructions, and only some of Github's
randomly assigned machines do which explains the
random failures. CPU capabilities can be queried using Numpy's new
`show_runtime` method, and absence of extensions AVX512F, AVX512CD and
AVX512_SKX correlates 100% with passing tests and vice versa.
The problem in sqrt is caused by the fact the `evaluable.Sqrt` evaluates
using `numpy.power`, but compares the result with `numpy.sqrt`, and the
vectorized routines result in slightly different results causing the
test to miss its tolerance threshold. The log2 test, likewise, is
evaluated differently than its reference. This PR fixes the tolerance
issue by setting the reference result equal to the way the operation is
evaluated, so that the tolerance is always reached regardless of CPU.

What remains unclear is why this problem arose with the release of Numpy
1.24, as the AVX-512 code paths were merged prior to version 1.22. Several
AVX-512 related commits have followed since, but it is not clear which
caused the particular differences that caused these tests to fail. Once
we have local access to a machine that supports AVX-512 extensions we
can bisect Numpy to find the culprit and perhaps understand the true
source of the numerical change (however slight), but in the meantime we
can conclude that there is no reason to maintain the ban on Numpy 1.24.
  • Loading branch information
gertjanvanzwieten committed Jan 30, 2023
2 parents 6282b78 + f442f7a commit 0b2acb1
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 3 deletions.
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ requires-python = '~=3.7'
dependencies = [
"appdirs~=1.0",
"bottombar~=2.0.2",
"numpy>=1.17,<1.24",
"numpy>=1.17",
"nutils-poly~=1.0",
"psutil~=5.0",
"stringly",
Expand Down
4 changes: 2 additions & 2 deletions tests/test_evaluable.py
Original file line number Diff line number Diff line change
Expand Up @@ -454,12 +454,12 @@ def _check(name, op, n_op, *arg_values, hasgrad=True, zerograd=False, ndim=2):
_check('cos-complex', evaluable.cos, numpy.cos, ANC(4, 4))
_check('tan', evaluable.tan, numpy.tan, ANY(4, 4))
_check('tan-complex', evaluable.tan, numpy.tan, ANC(4, 4))
_check('sqrt', evaluable.sqrt, numpy.sqrt, NN(4, 4))
_check('sqrt', evaluable.sqrt, lambda a: numpy.power(a, .5), NN(4, 4)) # NOTE: not comparing against numpy.sqrt because of AVX-512 related accuracy issues, see #770
_check('sqrt-complex', evaluable.sqrt, numpy.sqrt, ANC(4, 4))
_check('log', evaluable.ln, numpy.log, POS(2, 2))
_check('log-complex', evaluable.ln, numpy.log, NZC(2, 2))
_check('log2', evaluable.log2, numpy.log2, POS(2, 2))
_check('log2-complex', evaluable.log2, numpy.log2, NZC(2, 2))
_check('log2-complex', evaluable.log2, lambda a: numpy.log(a) * numpy.power(numpy.log(2), -1), NZC(2, 2)) # NOTE: not comparing against numpy.log2 because of AVX-512 related accuracy issues, see #770
_check('log10', evaluable.log10, numpy.log10, POS(2, 2))
_check('log10-complex', evaluable.log10, numpy.log10, NZC(2, 2))
_check('exp', evaluable.exp, numpy.exp, ANY(4, 4))
Expand Down

0 comments on commit 0b2acb1

Please sign in to comment.