Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DOC, MAINT: smoke-docs failures on MacOS ARM + accelerate + NumPy 2.0.0rc2 #20960

Open
tylerjereddy opened this issue Jun 14, 2024 · 11 comments
Open
Labels
Documentation Issues related to the SciPy documentation. Also check https://github.com/scipy/scipy.org maintenance Items related to regular maintenance tasks

Comments

@tylerjereddy
Copy link
Contributor

Request for report from gh-20933. On latest main (d304e95) on MacOS ARM with Accelerate, Python 3.12, NumPy 2.0.0rc2. Some failures are linalg things that may not be surprising with a backend swap, others are sparse + NumPy 2 related I believe, and were very briefly discussed with CJ somewhere else I think.

  • python dev.py build -j 16 --with-accelerate
  • python dev.py -n smoke-docs
Error Output
============================================================================================ FAILURES =============================================================================================
___________________________________________________________________________________ [doctest] scipy.io.hb_read ____________________________________________________________________________________
491 
492     Examples
493     --------
494     We can read and write a harwell-boeing format file:
495 
496     >>> from scipy.io import hb_read, hb_write
497     >>> from scipy.sparse import csr_array, eye
498     >>> data = csr_array(eye(3))  # create a sparse array
499     >>> hb_write("data.hb", data)  # write a hb file
500     >>> print(hb_read("data.hb"))  # read a hb file
Differences (unified diff with -expected +actual):
    @@ -1,3 +1,6 @@
    -(np.int32(0), np.int32(0))  1.0
    -(np.int32(1), np.int32(1))  1.0
    -(np.int32(2), np.int32(2))  1.0
    +<Compressed Sparse Column sparse matrix of dtype 'float64'
    +	with 3 stored elements and shape (3, 3)>
    +  Coords	Values
    +  (0, 0)	1.0
    +  (1, 1)	1.0
    +  (2, 2)	1.0

/Users/treddy/github_projects/scipy_2/build-install/lib/python3.12/site-packages/scipy/io/_harwell_boeing/hb.py:500: DocTestFailure
___________________________________________________________________________________ [doctest] scipy.io.hb_write ___________________________________________________________________________________
541 
542     Examples
543     --------
544     We can read and write a harwell-boeing format file:
545 
546     >>> from scipy.io import hb_read, hb_write
547     >>> from scipy.sparse import csr_array, eye
548     >>> data = csr_array(eye(3))  # create a sparse array
549     >>> hb_write("data.hb", data)  # write a hb file
550     >>> print(hb_read("data.hb"))  # read a hb file
Differences (unified diff with -expected +actual):
    @@ -1,3 +1,6 @@
    -(np.int32(0), np.int32(0))  1.0
    -(np.int32(1), np.int32(1))  1.0
    -(np.int32(2), np.int32(2))  1.0
    +<Compressed Sparse Column sparse matrix of dtype 'float64'
    +	with 3 stored elements and shape (3, 3)>
    +  Coords	Values
    +  (0, 0)	1.0
    +  (1, 1)	1.0
    +  (2, 2)	1.0

/Users/treddy/github_projects/scipy_2/build-install/lib/python3.12/site-packages/scipy/io/_harwell_boeing/hb.py:550: DocTestFailure
__________________________________________________________________________________ [doctest] scipy.linalg.schur ___________________________________________________________________________________
100     array([[0.72711591, -0.60156188, 0.33079564],
101            [0.52839428, 0.79801892, 0.28976765],
102            [0.43829436, 0.03590414, -0.89811411]])
103 
104     >>> T2, Z2 = schur(A, output='complex')
105     >>> T2
106     array([[ 2.65896708, -1.22839825+1.32378589j,  0.42590089+1.51937378j], # may vary
107            [ 0.        , -0.32948354+0.80225456j, -0.59877807+0.56192146j],
108            [ 0.        ,  0.                    , -0.32948354-0.80225456j]])
109     >>> eigvals(T2)
Expected:
    array([2.65896708, -0.32948354+0.80225456j, -0.32948354-0.80225456j])
Got:
    array([ 2.65896708+0.j        , -0.32948354-0.80225456j,
           -0.32948354+0.80225456j])

/Users/treddy/github_projects/scipy_2/build-install/lib/python3.12/site-packages/scipy/linalg/_decomp_schur.py:109: DocTestFailure
_______________________________________________________________________ [doctest] scipy.sparse.csgraph.connected_components _______________________________________________________________________
EXAMPLE LOCATION UNKNOWN, not showing all tests of that example
??? >>> print(graph)
Differences (unified diff with -expected +actual):
    @@ -1,4 +1,7 @@
    -  (np.int32(0), np.int32(1))        1
    -  (np.int32(0), np.int32(2))        1
    -  (np.int32(1), np.int32(2))        1
    -  (np.int32(3), np.int32(4))        1
    +<Compressed Sparse Row sparse matrix of dtype 'int64'
    +	with 4 stored elements and shape (5, 5)>
    +  Coords	Values
    +  (0, 1)	1
    +  (0, 2)	1
    +  (1, 2)	1
    +  (3, 4)	1

/Users/treddy/github_projects/scipy_2/build-install/lib/python3.12/site-packages/scipy/sparse/csgraph/_traversal.cpython-312-darwin.so:None: DocTestFailure
__________________________________________________________________________ [doctest] scipy.sparse.csgraph.shortest_path ___________________________________________________________________________
EXAMPLE LOCATION UNKNOWN, not showing all tests of that example
??? >>> print(graph)
Differences (unified diff with -expected +actual):
    @@ -1,5 +1,8 @@
    -  (np.int32(0), np.int32(1))        1
    -  (np.int32(0), np.int32(2))        2
    -  (np.int32(1), np.int32(3))        1
    -  (np.int32(2), np.int32(0))        2
    -  (np.int32(2), np.int32(3))        3
    +<Compressed Sparse Row sparse matrix of dtype 'int64'
    +	with 5 stored elements and shape (4, 4)>
    +  Coords	Values
    +  (0, 1)	1
    +  (0, 2)	2
    +  (1, 3)	1
    +  (2, 0)	2
    +  (2, 3)	3

/Users/treddy/github_projects/scipy_2/build-install/lib/python3.12/site-packages/scipy/sparse/csgraph/_shortest_path.cpython-312-darwin.so:None: DocTestFailure
__________________________________________________________________________ [doctest] scipy.sparse.csgraph.floyd_warshall __________________________________________________________________________
EXAMPLE LOCATION UNKNOWN, not showing all tests of that example
??? >>> print(graph)
Differences (unified diff with -expected +actual):
    @@ -1,5 +1,8 @@
    -  (np.int32(0), np.int32(1))        1
    -  (np.int32(0), np.int32(2))        2
    -  (np.int32(1), np.int32(3))        1
    -  (np.int32(2), np.int32(0))        2
    -  (np.int32(2), np.int32(3))        3
    +<Compressed Sparse Row sparse matrix of dtype 'int64'
    +	with 5 stored elements and shape (4, 4)>
    +  Coords	Values
    +  (0, 1)	1
    +  (0, 2)	2
    +  (1, 3)	1
    +  (2, 0)	2
    +  (2, 3)	3

/Users/treddy/github_projects/scipy_2/build-install/lib/python3.12/site-packages/scipy/sparse/csgraph/_shortest_path.cpython-312-darwin.so:None: DocTestFailure
_____________________________________________________________________________ [doctest] scipy.sparse.csgraph.dijkstra _____________________________________________________________________________
EXAMPLE LOCATION UNKNOWN, not showing all tests of that example
??? >>> print(graph)
Differences (unified diff with -expected +actual):
    @@ -1,4 +1,7 @@
    -  (np.int32(0), np.int32(1))        1
    -  (np.int32(0), np.int32(2))        2
    -  (np.int32(1), np.int32(3))        1
    -  (np.int32(2), np.int32(3))        3
    +<Compressed Sparse Row sparse matrix of dtype 'int64'
    +	with 4 stored elements and shape (4, 4)>
    +  Coords	Values
    +  (0, 1)	1
    +  (0, 2)	2
    +  (1, 3)	1
    +  (2, 3)	3

/Users/treddy/github_projects/scipy_2/build-install/lib/python3.12/site-packages/scipy/sparse/csgraph/_shortest_path.cpython-312-darwin.so:None: DocTestFailure
___________________________________________________________________________ [doctest] scipy.sparse.csgraph.bellman_ford ___________________________________________________________________________
EXAMPLE LOCATION UNKNOWN, not showing all tests of that example
??? >>> print(graph)
Differences (unified diff with -expected +actual):
    @@ -1,5 +1,8 @@
    -  (np.int32(0), np.int32(1))        1
    -  (np.int32(0), np.int32(2))        2
    -  (np.int32(1), np.int32(3))        1
    -  (np.int32(2), np.int32(0))        2
    -  (np.int32(2), np.int32(3))        3
    +<Compressed Sparse Row sparse matrix of dtype 'int64'
    +	with 5 stored elements and shape (4, 4)>
    +  Coords	Values
    +  (0, 1)	1
    +  (0, 2)	2
    +  (1, 3)	1
    +  (2, 0)	2
    +  (2, 3)	3

/Users/treddy/github_projects/scipy_2/build-install/lib/python3.12/site-packages/scipy/sparse/csgraph/_shortest_path.cpython-312-darwin.so:None: DocTestFailure
_____________________________________________________________________________ [doctest] scipy.sparse.csgraph.johnson ______________________________________________________________________________
EXAMPLE LOCATION UNKNOWN, not showing all tests of that example
??? >>> print(graph)
Differences (unified diff with -expected +actual):
    @@ -1,5 +1,8 @@
    -  (np.int32(0), np.int32(1))        1
    -  (np.int32(0), np.int32(2))        2
    -  (np.int32(1), np.int32(3))        1
    -  (np.int32(2), np.int32(0))        2
    -  (np.int32(2), np.int32(3))        3
    +<Compressed Sparse Row sparse matrix of dtype 'int64'
    +	with 5 stored elements and shape (4, 4)>
    +  Coords	Values
    +  (0, 1)	1
    +  (0, 2)	2
    +  (1, 3)	1
    +  (2, 0)	2
    +  (2, 3)	3

/Users/treddy/github_projects/scipy_2/build-install/lib/python3.12/site-packages/scipy/sparse/csgraph/_shortest_path.cpython-312-darwin.so:None: DocTestFailure
_______________________________________________________________________________ [doctest] scipy.sparse.csgraph.yen ________________________________________________________________________________
EXAMPLE LOCATION UNKNOWN, not showing all tests of that example
??? >>> print(graph)
Differences (unified diff with -expected +actual):
    @@ -1,5 +1,8 @@
    -  (np.int32(0), np.int32(1))        1
    -  (np.int32(0), np.int32(2))        2
    -  (np.int32(1), np.int32(3))        1
    -  (np.int32(2), np.int32(0))        2
    -  (np.int32(2), np.int32(3))        3
    +<Compressed Sparse Row sparse matrix of dtype 'int64'
    +	with 5 stored elements and shape (4, 4)>
    +  Coords	Values
    +  (0, 1)	1
    +  (0, 2)	2
    +  (1, 3)	1
    +  (2, 0)	2
    +  (2, 3)	3

/Users/treddy/github_projects/scipy_2/build-install/lib/python3.12/site-packages/scipy/sparse/csgraph/_shortest_path.cpython-312-darwin.so:None: DocTestFailure
_______________________________________________________________________ [doctest] scipy.sparse.csgraph.breadth_first_order ________________________________________________________________________
EXAMPLE LOCATION UNKNOWN, not showing all tests of that example
??? >>> print(graph)
Differences (unified diff with -expected +actual):
    @@ -1,5 +1,8 @@
    -  (np.int32(0), np.int32(1))        1
    -  (np.int32(0), np.int32(2))        2
    -  (np.int32(1), np.int32(3))        1
    -  (np.int32(2), np.int32(0))        2
    -  (np.int32(2), np.int32(3))        3
    +<Compressed Sparse Row sparse matrix of dtype 'int64'
    +	with 5 stored elements and shape (4, 4)>
    +  Coords	Values
    +  (0, 1)	1
    +  (0, 2)	2
    +  (1, 3)	1
    +  (2, 0)	2
    +  (2, 3)	3

/Users/treddy/github_projects/scipy_2/build-install/lib/python3.12/site-packages/scipy/sparse/csgraph/_traversal.cpython-312-darwin.so:None: DocTestFailure
________________________________________________________________________ [doctest] scipy.sparse.csgraph.depth_first_order _________________________________________________________________________
EXAMPLE LOCATION UNKNOWN, not showing all tests of that example
??? >>> print(graph)
Differences (unified diff with -expected +actual):
    @@ -1,5 +1,8 @@
    -(np.int32(0), np.int32(1))  1
    -(np.int32(0), np.int32(2))  2
    -(np.int32(1), np.int32(3))  1
    -(np.int32(2), np.int32(0))  2
    -(np.int32(2), np.int32(3))  3
    +<Compressed Sparse Row sparse matrix of dtype 'int64'
    +	with 5 stored elements and shape (4, 4)>
    +  Coords	Values
    +  (0, 1)	1
    +  (0, 2)	2
    +  (1, 3)	1
    +  (2, 0)	2
    +  (2, 3)	3

/Users/treddy/github_projects/scipy_2/build-install/lib/python3.12/site-packages/scipy/sparse/csgraph/_traversal.cpython-312-darwin.so:None: DocTestFailure
______________________________________________________________________ [doctest] scipy.sparse.csgraph.reverse_cuthill_mckee _______________________________________________________________________
EXAMPLE LOCATION UNKNOWN, not showing all tests of that example
??? >>> print(graph)
Differences (unified diff with -expected +actual):
    @@ -1,5 +1,8 @@
    -  (np.int32(0), np.int32(1))        1
    -  (np.int32(0), np.int32(2))        2
    -  (np.int32(1), np.int32(3))        1
    -  (np.int32(2), np.int32(0))        2
    -  (np.int32(2), np.int32(3))        3
    +<Compressed Sparse Row sparse matrix of dtype 'int64'
    +	with 5 stored elements and shape (4, 4)>
    +  Coords	Values
    +  (0, 1)	1
    +  (0, 2)	2
    +  (1, 3)	1
    +  (2, 0)	2
    +  (2, 3)	3

/Users/treddy/github_projects/scipy_2/build-install/lib/python3.12/site-packages/scipy/sparse/csgraph/_reordering.cpython-312-darwin.so:None: DocTestFailure
_________________________________________________________________________ [doctest] scipy.sparse.csgraph.structural_rank __________________________________________________________________________
EXAMPLE LOCATION UNKNOWN, not showing all tests of that example
??? >>> print(graph)
Differences (unified diff with -expected +actual):
    @@ -1,8 +1,11 @@
    -  (np.int32(0), np.int32(1))        1
    -  (np.int32(0), np.int32(2))        2
    -  (np.int32(1), np.int32(0))        1
    -  (np.int32(1), np.int32(3))        1
    -  (np.int32(2), np.int32(0))        2
    -  (np.int32(2), np.int32(3))        3
    -  (np.int32(3), np.int32(1))        1
    -  (np.int32(3), np.int32(2))        3
    +<Compressed Sparse Row sparse matrix of dtype 'int64'
    +	with 8 stored elements and shape (4, 4)>
    +  Coords	Values
    +  (0, 1)	1
    +  (0, 2)	2
    +  (1, 0)	1
    +  (1, 3)	1
    +  (2, 0)	2
    +  (2, 3)	3
    +  (3, 1)	1
    +  (3, 2)	3

/Users/treddy/github_projects/scipy_2/build-install/lib/python3.12/site-packages/scipy/sparse/csgraph/_reordering.cpython-312-darwin.so:None: DocTestFailure
______________________________________________________________________ [doctest] scipy.sparse.csgraph.construct_dist_matrix _______________________________________________________________________
EXAMPLE LOCATION UNKNOWN, not showing all tests of that example
??? >>> print(graph)
Differences (unified diff with -expected +actual):
    @@ -1,4 +1,7 @@
    -   (np.int32(0), np.int32(1))       1
    -   (np.int32(0), np.int32(2))       2
    -   (np.int32(1), np.int32(3))       1
    -   (np.int32(2), np.int32(3))       3
    +<Compressed Sparse Row sparse matrix of dtype 'int64'
    +	with 4 stored elements and shape (4, 4)>
    +  Coords	Values
    +  (0, 1)	1
    +  (0, 2)	2
    +  (1, 3)	1
    +  (2, 3)	3

/Users/treddy/github_projects/scipy_2/build-install/lib/python3.12/site-packages/scipy/sparse/csgraph/_tools.cpython-312-darwin.so:None: DocTestFailure
_________________________________________________________________________ [doctest] scipy.sparse.csgraph.reconstruct_path _________________________________________________________________________
EXAMPLE LOCATION UNKNOWN, not showing all tests of that example
??? >>> print(graph)
Differences (unified diff with -expected +actual):
    @@ -1,4 +1,7 @@
    -   (np.int32(0), np.int32(1))       1
    -   (np.int32(0), np.int32(2))       2
    -   (np.int32(1), np.int32(3))       1
    -   (np.int32(2), np.int32(3))       3
    +<Compressed Sparse Row sparse matrix of dtype 'int64'
    +	with 4 stored elements and shape (4, 4)>
    +  Coords	Values
    +  (0, 1)	1
    +  (0, 2)	2
    +  (1, 3)	1
    +  (2, 3)	3

/Users/treddy/github_projects/scipy_2/build-install/lib/python3.12/site-packages/scipy/sparse/csgraph/_tools.cpython-312-darwin.so:None: DocTestFailure
===================================================================================== short test summary info =====================================================================================
FAILED scipy/io/__init__.py::scipy.io.hb_read
FAILED scipy/io/__init__.py::scipy.io.hb_write
FAILED scipy/linalg/__init__.py::scipy.linalg.schur
FAILED scipy/sparse/csgraph/__init__.py::scipy.sparse.csgraph.connected_components
FAILED scipy/sparse/csgraph/__init__.py::scipy.sparse.csgraph.shortest_path
FAILED scipy/sparse/csgraph/__init__.py::scipy.sparse.csgraph.floyd_warshall
FAILED scipy/sparse/csgraph/__init__.py::scipy.sparse.csgraph.dijkstra
FAILED scipy/sparse/csgraph/__init__.py::scipy.sparse.csgraph.bellman_ford
FAILED scipy/sparse/csgraph/__init__.py::scipy.sparse.csgraph.johnson
FAILED scipy/sparse/csgraph/__init__.py::scipy.sparse.csgraph.yen
FAILED scipy/sparse/csgraph/__init__.py::scipy.sparse.csgraph.breadth_first_order
FAILED scipy/sparse/csgraph/__init__.py::scipy.sparse.csgraph.depth_first_order
FAILED scipy/sparse/csgraph/__init__.py::scipy.sparse.csgraph.reverse_cuthill_mckee
FAILED scipy/sparse/csgraph/__init__.py::scipy.sparse.csgraph.structural_rank
FAILED scipy/sparse/csgraph/__init__.py::scipy.sparse.csgraph.construct_dist_matrix
FAILED scipy/sparse/csgraph/__init__.py::scipy.sparse.csgraph.reconstruct_path
=========================================================================== 16 failed, 1296 passed in 73.12s (0:01:13) ============================================================================
@tylerjereddy tylerjereddy added the Documentation Issues related to the SciPy documentation. Also check https://github.com/scipy/scipy.org label Jun 14, 2024
@github-actions github-actions bot added the maintenance Items related to regular maintenance tasks label Jun 14, 2024
@ev-br
Copy link
Member

ev-br commented Jun 14, 2024

Agreed, sparse / csgraph is numpy 2 related. Also the way sparse arrays print is not very doctester-friendly (how to fish numerals or tuples from that repr?) and it falls back to the vanilla doctest. Possible ways out:

  • try to do >>> graph instead of >>> print(graph) [unlikely to be helpful]
  • tweak examples to be more doctester-friendly, like printing the values/indices separately [yikes]
  • just wait until numpy 2 is the default

The svdvals thing is indeed expected. lapack does not guarantee the order of complex conjugate eigenvalues. Sorting complex arrays gets ugly quick (we have it in the tests somewhere), so

  • change the example to have only real arrays
  • add # may vary (order) (the part in brackets can be changed freely)

@ev-br
Copy link
Member

ev-br commented Jun 15, 2024

Yet another option is to conditionally add entries to a skiplist :

https://github.com/scipy/scipy/blob/main/scipy/conftest.py#L364

@rgommers
Copy link
Member

The preferred solutions I think are:

  • just wait until numpy 2 is the default
  • add # may vary (order)

ev-br added a commit to ev-br/scipy that referenced this issue Jun 16, 2024
The ordering of complex conjugate pairs is not guaranteed and
was reported to differ in scipy#20960
ev-br added a commit to ev-br/scipy that referenced this issue Jun 16, 2024
The ordering of complex conjugate pairs is not guaranteed and
was reported to differ in scipy#20960
@rgommers
Copy link
Member

Other issues:

(1) a too obscure error if scipy-doctest isn't installed:

python dev.py smoke-docs
💻  ninja -C /Users/rgommers/code/scipy/build -j8
ninja: Entering directory `/Users/rgommers/code/scipy/build'
[2/2] Generating scipy/generate-version with a custom command
Build OK
💻  meson install -C build --only-changed --tags runtime,python-runtime,tests,devel
Installing, see meson-install.log...
Installation OK
SciPy from development installed path at: /Users/rgommers/code/scipy/build-install/lib/python3.10/site-packages
Running tests for scipy version:1.15.0.dev0+949.89a855b, installed at:/Users/rgommers/code/scipy/build-install/lib/python3.10/site-packages/scipy
ERROR: usage: dev.py [options] [file_or_dir] [file_or_dir] [...]
dev.py: error: unrecognized arguments: --doctest-collect=api
  inifile: /Users/rgommers/code/scipy/pytest.ini
  rootdir: /Users/rgommers/code/scipy

(2) a hard error when the optional test dependency mpmath isn't installed:

...
ERROR scipy/special/_precompute/cosine_cdf.py - ModuleNotFoundError: No module named 'mpmath'
ERROR scipy/special/_precompute/struve_convergence.py - ModuleNotFoundError: No module named 'mpmath'
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection

@ev-br
Copy link
Member

ev-br commented Jun 18, 2024

(2) is not fixable, and should not be seen if run under dev.py; what was the command you ran?

(1) I'll send a PR with a fix

@rgommers
Copy link
Member

(2) is not fixable, and should not be seen if run under dev.py; what was the command you ran?

Same, python dev.py smoke-docs. And sure it's fixable - one way is to add scipy/special/_precompute to norecursedirs in pytest.py. There's nothing public in that submodule nor does it contain any tests, so no need to trawl through it.

@ev-br
Copy link
Member

ev-br commented Jun 18, 2024

Meant to mean:

  • is not fixable if a source with an optional import is fed to pytest explicitly
  • special/_precompute should already be in the ignore list, so the question is why it I'd correctly ignored for me locally and in the CI but not for you.

@rgommers
Copy link
Member

It's not in the pytest.ini list. If there's another list somewhere, can you please point to it? mpmath is installed in the base job in CircleCI, so I assume that explains that.

Another fix is to make the import conditional in the source files. I don't really mind which way it's fixed, but the current behavior is definitely wrong on main.

@ev-br
Copy link
Member

ev-br commented Jun 18, 2024

https://github.com/scipy/scipy/blob/main/scipy/_lib/_testutils.py#L98

(Maybe this list should go to conftest.py, to follow numpy; I'll be able to take a look later today)

@ev-br
Copy link
Member

ev-br commented Jun 21, 2024

cf #21017 for a fix for (2)
(1) takes a little longer, need to sort out register_assertion_rewrite pytest voodoo. Is on my TODO list though.

@ev-br
Copy link
Member

ev-br commented Jun 22, 2024

(1) should also be fixed by #21017 + an update to scipy_doctest==1.3, which is on pypi now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Documentation Issues related to the SciPy documentation. Also check https://github.com/scipy/scipy.org maintenance Items related to regular maintenance tasks
Projects
None yet
Development

No branches or pull requests

3 participants