Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Potential regression in libopenblas 0.3.20 for M1 #8096

Closed
esc opened this issue May 23, 2022 · 0 comments · Fixed by #8097
Closed

Potential regression in libopenblas 0.3.20 for M1 #8096

esc opened this issue May 23, 2022 · 0 comments · Fixed by #8097
Labels
bug - build/packaging Bugs: issue with building or packaging Numba

Comments

@esc
Copy link
Member

esc commented May 23, 2022

While testing Numba on M1 (osx-arm64) using the Anaconda internal build farm, the following errors started to re-appear:

INFO - ======================================================================
INFO - ERROR: test_linalg_eig (numba.tests.test_linalg.TestLinalgEigenSystems)
INFO - ----------------------------------------------------------------------
INFO - Traceback (most recent call last):
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numba/tests/test_linalg.py", line 1063, in test_linalg_eig
INFO -     self.checker_for_linalg_eig("eig", eig_matrix, 2, True)
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numba/tests/test_linalg.py", line 1013, in checker_for_linalg_eig
INFO -     check(a)
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numba/tests/test_linalg.py", line 907, in check
INFO -     got = cfunc(*args)
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numba/np/linalg.py", line 879, in _handle_err_maybe_convergence_problem
INFO -     raise ValueError("Internal algorithm failed to converge.")
INFO - ValueError: Internal algorithm failed to converge.
INFO - 
INFO - ======================================================================
INFO - FAIL: test_linalg_eigh (numba.tests.test_linalg.TestLinalgEigenSystems)
INFO - ----------------------------------------------------------------------
INFO - Traceback (most recent call last):
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numba/tests/test_linalg.py", line 1071, in test_linalg_eigh
INFO -     self.checker_for_linalg_eig("eigh", eigh_matrix, 2, False)
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numba/tests/test_linalg.py", line 1013, in checker_for_linalg_eig
INFO -     check(a)
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numba/tests/test_linalg.py", line 968, in check
INFO -     np.testing.assert_allclose(
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numpy/testing/_private/utils.py", line 1530, in assert_allclose
INFO -     assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numpy/testing/_private/utils.py", line 844, in assert_array_compare
INFO -     raise AssertionError(msg)
INFO - AssertionError:
INFO - Not equal to tolerance rtol=5e-06, atol=5e-06
INFO - 
INFO - Mismatched elements: 44 / 100 (44%)
INFO - Max absolute difference: 22.505447
INFO - Max relative difference: 1.1422548
INFO -  x: array([[-2.054423e-02, -1.056475e+00,  7.497622e-01,  1.002695e-01,
INFO -          9.463690e-01, -1.117992e+00,  4.027816e-01,  5.888832e-02,
INFO -         -4.328386e-03,  1.127720e-04],...
INFO -  y: array([[-2.054431e-02, -1.056475e+00,  7.497622e-01,  1.002698e-01,
INFO -          9.463690e-01, -1.117992e+00,  4.027817e-01,  5.888832e-02,
INFO -         -4.328387e-03,  1.128214e-04],...
INFO - 
INFO - ======================================================================
INFO - FAIL: test_linalg_eigvals (numba.tests.test_linalg.TestLinalgEigenSystems)
INFO - ----------------------------------------------------------------------
INFO - Traceback (most recent call last):
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numba/tests/test_linalg.py", line 1067, in test_linalg_eigvals
INFO -     self.checker_for_linalg_eig("eigvals", eigvals_matrix, 1, True)
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numba/tests/test_linalg.py", line 1013, in checker_for_linalg_eig
INFO -     check(a)
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numba/tests/test_linalg.py", line 987, in check
INFO -     np.testing.assert_allclose(
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numpy/testing/_private/utils.py", line 1530, in assert_allclose
INFO -     assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numpy/testing/_private/utils.py", line 844, in assert_array_compare
INFO -     raise AssertionError(msg)
INFO - AssertionError:
INFO - Not equal to tolerance rtol=5e-06, atol=5e-06
INFO - 
INFO - Mismatched elements: 10 / 10 (100%)
INFO - Max absolute difference: 4.7390566e+11
INFO - Max relative difference: 1.0013683
INFO -  x: array([-5.321849+1.040691j, -2.407394-0.067465j, -0.832093-0.659673j,
INFO -         0.654487+0.408842j,  1.334701+3.037374j,  2.917458+1.09749j ,
INFO -         5.619255+0.487094j,  9.144733-0.65504j , 13.739012-2.328744j,
INFO -        20.15169 -4.860569j], dtype=complex64)
INFO -  y: array([-1.651082e+11+4.442138e+11j, -1.370451e+10+3.778335e+10j,
INFO -        -1.148686e+09+3.095466e+09j, -7.053805e+07+1.893135e+08j,
INFO -        -1.227380e+06+4.060472e+06j, -2.457228e+04+6.749866e+04j,...
INFO - 
INFO - ======================================================================
INFO - FAIL: test_linalg_matrix_rank (numba.tests.test_linalg.TestLinalgMatrixRank)
INFO - Test np.linalg.matrix_rank
INFO - ----------------------------------------------------------------------
INFO - Traceback (most recent call last):
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numba/tests/test_linalg.py", line 2334, in test_linalg_matrix_rank
INFO -     self.assertEqual(cfunc(a, tol), 0)
INFO - AssertionError: 1 != 0
INFO - 
INFO - ======================================================================
INFO - FAIL: test_linalg_lstsq (numba.tests.test_linalg.TestLinalgLstsq)
INFO - Test np.linalg.lstsq
INFO - ----------------------------------------------------------------------
INFO - Traceback (most recent call last):
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numba/tests/test_linalg.py", line 1461, in check
INFO -     np.testing.assert_allclose(
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numpy/testing/_private/utils.py", line 1530, in assert_allclose
INFO -     assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numpy/testing/_private/utils.py", line 844, in assert_array_compare
INFO -     raise AssertionError(msg)
INFO - AssertionError:
INFO - Not equal to tolerance rtol=1e-05, atol=1e-05
INFO - 
INFO - Mismatched elements: 3 / 3 (100%)
INFO - Max absolute difference: 0.74847513
INFO - Max relative difference: 0.48853713
INFO -  x: array([[0.+2.j ],
INFO -        [1.+1.5j],
INFO -        [2.+1.j ]], dtype=complex64)
INFO -  y: array([[0.3631  +2.153484j],
INFO -        [1.10943 +1.050372j],
INFO -        [1.451278+0.490963j]], dtype=complex64)
INFO - 
INFO - During handling of the above exception, another exception occurred:
INFO - 
INFO - Traceback (most recent call last):
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numba/tests/test_linalg.py", line 1473, in check
INFO -     np.testing.assert_allclose(
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numpy/testing/_private/utils.py", line 1530, in assert_allclose
INFO -     assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numpy/testing/_private/utils.py", line 844, in assert_array_compare
INFO -     raise AssertionError(msg)
INFO - AssertionError:
INFO - Not equal to tolerance rtol=0.0001, atol=0.0001
INFO - 
INFO - Mismatched elements: 3 / 3 (100%)
INFO - Max absolute difference: 0.89143157
INFO - Max relative difference: 0.34606287
INFO -  x: array([[ 2.373896+1.891432j],
INFO -        [ 0.907289+1.24352j ],
INFO -        [-0.559317+0.595609j]], dtype=complex64)
INFO -  y: array([[ 2.373896+1.j      ],
INFO -        [ 0.722813+1.118777j],
INFO -        [-0.850073+0.564423j]], dtype=complex64)
INFO - 
INFO - During handling of the above exception, another exception occurred:
INFO - 
INFO - Traceback (most recent call last):
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numba/tests/test_linalg.py", line 1547, in test_linalg_lstsq
INFO -     inner_test_loop_fn(A, dt)
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numba/tests/test_linalg.py", line 1530, in inner_test_loop_fn
INFO -     check(A, B, **kwargs)
INFO -   File "/Users/ci/miniconda3-arm64/envs/testenv_6991686f-32eb-4cbb-a4f9-5a134ba5f7c3/lib/python3.9/site-packages/numba/tests/test_linalg.py", line 1482, in check
INFO -     self.assertGreater(10 * c, (1. / resolution))
INFO - AssertionError: 10.0 not greater than 1000000.0025247573

These were initially present with libopenblas 0.3.17 as reported in:

#7822 (comment)

The appear fixed with 0.3.18 (we don't see the failures) but now re-appear with 0.3.20.
@esc esc added the bug - build/packaging Bugs: issue with building or packaging Numba label May 23, 2022
@esc esc mentioned this issue May 23, 2022
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug - build/packaging Bugs: issue with building or packaging Numba
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Exclude libopenblas 0.3.20 on osx-arm64 esc/numba
1 participant
@esc