Test failures with OpenBLAS 0.3.26

[risicle]

Hi, just a heads-up here - in bumping our OpenBLAS package in NixOS to 0.3.26 (from 0.3.25), we've noticed some new scipy test failures with the error _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword liwork: dsyevr:liwork=1.

Full errors @ https://gist.github.com/risicle/5e47926a7a1f2019d9c8f596a317a0e1

This looks like it might be related to the release notes item:

• handle miscalculation of minimum work array size in corner cases (Reference-LAPACK PR 942)

whose trail leads to Reference-LAPACK/lapack#942

This is with numpy 1.26.2, scipy 1.11.4, python 3.11, gcc 13.2 x86_64 linux. Full system could reproduced from branch in PR NixOS/nixpkgs#278429 if necessary.

[rgommers]

Thanks for the report @risicle. Looks like there's something to fix in our f2py signature files.

Reference-LAPACK/lapack#942 is massive, I'm not sure that that is easy to go through - probably better to start from the failing tests, and figuring out what LAPACK routines are being called there, then checking their lwork definitions.

Copying from the gist for ease of checking:

lib/python3.11/site-packages/scipy/linalg/tests/test_decomp_polar.py::�[1mtest_verify_cases�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/sparse/linalg/_eigen/lobpcg/tests/test_lobpcg.py::�[1mtest_inplace_warning�[0m - _flapack.error: (lrwork>=max(24*n,1)||lrwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/sparse/linalg/_eigen/tests/test_svds.py::�[1mTest_SVDS_LOBPCG::test_svd_v0�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/sparse/linalg/_eigen/tests/test_svds.py::�[1mTest_SVDS_LOBPCG::test_svd_random_state�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/sparse/linalg/_eigen/tests/test_svds.py::�[1mTest_SVDS_LOBPCG::test_svd_random_state_2[0]�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/sparse/linalg/_eigen/tests/test_svds.py::�[1mTest_SVDS_LOBPCG::test_svd_random_state_2[1]�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/sparse/linalg/_eigen/tests/test_svds.py::�[1mTest_SVDS_LOBPCG::test_svd_random_state_2[random_state2]�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/sparse/linalg/_eigen/tests/test_svds.py::�[1mTest_SVDS_LOBPCG::test_svd_random_state_2[random_state3]�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/sparse/linalg/_eigen/tests/test_svds.py::�[1mTest_SVDS_LOBPCG::test_svd_maxiter�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/sparse/linalg/_eigen/lobpcg/tests/test_lobpcg.py::�[1mtest_regression�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/sparse/linalg/_eigen/lobpcg/tests/test_lobpcg.py::�[1mtest_failure_to_run_iterations_nonsymmetric�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/sparse/linalg/_eigen/lobpcg/tests/test_lobpcg.py::�[1mtest_hermitian�[0m - _flapack.error: (lrwork>=max(24*n,1)||lrwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/sparse/linalg/_eigen/lobpcg/tests/test_lobpcg.py::�[1mtest_verbosity�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multicomp.py::�[1mTestDunnett::test_critical_values[0.5-1-10-2.23-0.05-two-sided]�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multicomp.py::�[1mTestDunnett::test_critical_values[0.5-1-10-1.81-0.05-greater]�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multicomp.py::�[1mTestDunnett::test_ttest_ind[two-sided]�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMultivariateNormal::test_logpdf_default_values�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multicomp.py::�[1mTestDunnett::test_ttest_ind[less]�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMultivariateNormal::test_logcdf_default_values�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multicomp.py::�[1mTestDunnett::test_ttest_ind[greater]�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMultivariateNormal::test_degenerate_distributions�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMultivariateT::test_mvt_with_df_one_is_cauchy�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMultivariateNormal::test_normal_1D�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMultivariateT::test_mvt_with_high_df_is_approx_normal�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMultivariateNormal::test_rvs_shape�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMultivariateT::test_mvt_with_inf_df_calls_normal�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMultivariateT::test_shape_correctness�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mtest_random_state_property�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMatrixNormal::test_default_inputs�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMultivariateT::test_default_arguments�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMatrixNormal::test_frozen_matrix_normal�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMultivariateT::test_entropy_1d[1]�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMultivariateT::test_entropy_1d[10]�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMatrixNormal::test_matches_multivariate�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMultivariateT::test_default_args[None-None-None-0-1-1]�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMultivariateT::test_entropy_1d[100]�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMultivariateT::test_default_args[None-None-7-0-1-7]�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMultivariateT::test_scalar_list_and_ndarray_arguments[-1-2-3-loc_ans0-shape_ans0-3]�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multicomp.py::�[1mTestDunnett::test_shapes[1]�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMultivariateT::test_scalar_list_and_ndarray_arguments[loc1-shape1-3-loc_ans1-shape_ans1-3]�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMultivariateNormal::test_scalar_values�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMultivariateT::test_scalar_list_and_ndarray_arguments[loc2-shape2-3-loc_ans2-shape_ans2-3]�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMultivariateT::test_cdf_against_multivariate_normal[1]�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
lib/python3.11/site-packages/scipy/stats/tests/test_multivariate.py::�[1mTestMultivariateT::test_cdf_against_univariate_t�[0m - _flapack.error: (liwork>=max(1,10*n)||liwork==-1) failed for 10th keyword l...
44 failed�[0m, �[32m42429 passed�[0m, �[33m2825 skipped�[0m, �[33m135 xfailed�[0m, �[33m11 xpassed�[0m�[31m in 127.02s (0:02:07)�[0m�[31m =�[0m

[ilayn]

This is in the corner of my mind. I saw it in LAPACK repo, I was going to do something about it, then I went into special rewrite. If someone else doesn't jump in, I'll circle back to this.