test_viz.py failure with "ValueError: Masked arrays must be 1-D"

[emollier]

Good day,

I'm trying to upgrade nitime on Debian unstable to the version
tagged 0.10.1, but when I run the test suite, I'm getting the
following failure:

___________________________ test_drawgraph_channels ____________________________

   @pytest.mark.skipif(is_ci, reason="Running on a CI server")
   @pytest.mark.skipif(no_networkx, reason=no_networkx_msg)
   def test_drawgraph_channels():
>       fig04 = drawgraph_channels(C.corrcoef, roi_names)

nitime/tests/test_viz.py:52: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
nitime/viz.py:425: in drawgraph_channels
   fig = draw_graph(G,
nitime/viz.py:747: in draw_graph
   nx.draw_networkx_nodes(G,
/usr/lib/python3/dist-packages/networkx/drawing/nx_pylab.py:433: in draw_networkx_nodes
   node_collection = ax.scatter(
/usr/lib/python3/dist-packages/matplotlib/__init__.py:1437: in inner
   return func(ax, *map(sanitize_sequence, args), **kwargs)
/usr/lib/python3/dist-packages/matplotlib/axes/_axes.py:4554: in scatter
   cbook._combine_masks(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = (masked_array(data=[1.],
            mask=False,
      fill_value=1e+20), masked_array(data=[1.4702742e-08],
      ...        3052.78226457]],
             mask=False,
       fill_value=1e+20), 'w', array([[1., 1., 1., 1.]]), 'k', ...)
nrecs = 1
margs = [masked_array(data=[1.],
            mask=False,
      fill_value=1e+20), masked_array(data=[1.4702742e-08],
            mask=False,
      fill_value=1e+20)]
seqlist = [True, True, False, False, False, False, ...]

   def _combine_masks(*args):
       """
       Find all masked and/or non-finite points in a set of arguments,
       and return the arguments as masked arrays with a common mask.
   
       Arguments can be in any of 5 categories:
   
       1) 1-D masked arrays
       2) 1-D ndarrays
       3) ndarrays with more than one dimension
       4) other non-string iterables
       5) anything else
   
       The first argument must be in one of the first four categories;
       any argument with a length differing from that of the first
       argument (and hence anything in category 5) then will be
       passed through unchanged.
   
       Masks are obtained from all arguments of the correct length
       in categories 1, 2, and 4; a point is bad if masked in a masked
       array or if it is a nan or inf.  No attempt is made to
       extract a mask from categories 2 and 4 if `numpy.isfinite`
       does not yield a Boolean array.  Category 3 is included to
       support RGB or RGBA ndarrays, which are assumed to have only
       valid values and which are passed through unchanged.
   
       All input arguments that are not passed unchanged are returned
       as masked arrays if any masked points are found, otherwise as
       ndarrays.
   
       """
       if not len(args):
           return ()
       if is_scalar_or_string(args[0]):
           raise ValueError("First argument must be a sequence")
       nrecs = len(args[0])
       margs = []  # Output args; some may be modified.
       seqlist = [False] * len(args)  # Flags: True if output will be masked.
       masks = []    # List of masks.
       for i, x in enumerate(args):
           if is_scalar_or_string(x) or len(x) != nrecs:
               margs.append(x)  # Leave it unmodified.
           else:
               if isinstance(x, np.ma.MaskedArray) and x.ndim > 1:
>                   raise ValueError("Masked arrays must be 1-D")
E                   ValueError: Masked arrays must be 1-D

/usr/lib/python3/dist-packages/matplotlib/cbook/__init__.py:1094: ValueError

My test bed runs the following software versions:

• Linux 6.4
• Python 3.11.4
• pytest 7.4.0
• pluggy 1.2.0
• networkx 2.8.8
• matplotlib 3.6.3

In case that matters, I see the skips when running in continuous
integration context in test_viz.py, and it is quite possible my
test bed shares some similarities with such context. What was
the rationale, if that matters?

Have a nice day, :)
Étienne.

[nileshpatra]

This is a regression and the offending commit is ace0167. The function used is not a drop in replacement, as it renders a matrix instead of array.

Python 3.11.4 (main, Jun  7 2023, 10:13:09) [GCC 12.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import networkx
>>> import networkx as nx
>>> G1 = nx.Graph([(1, 1)])
>>> A1 = nx.adjacency_matrix(G1).todense()
<stdin>:1: FutureWarning: adjacency_matrix will return a scipy.sparse array instead of a matrix in Networkx 3.0.
>>> A1
matrix([[1]])
>>> A2 = nx.adj_matrix(G1).A
<stdin>:1: DeprecationWarning: adj_matrix is deprecated and will be removed in version 3.0.
Use `adjacency_matrix` instead

/usr/lib/python3/dist-packages/networkx/linalg/graphmatrix.py:187: FutureWarning: adjacency_matrix will return a scipy.sparse array instead of a matrix in Networkx 3.0.
  return adjacency_matrix(G, nodelist, dtype, weight)
>>> A2
array([[1]])

Probably adding a ".A" after todense() should be good enough.

/cc: @effigies

[effigies]

It looks like networkx changed behavior at 3.0, as adjacency_matrix now returns a sparse array. Can we instead just use G1.to_numpy_array() and get the same result across the supported networkx versions?

[emollier]

I think it would be feasible to use the to_numpy_array function.
If I extend Nilesh's example, then I get:

>>> nx.__version__
'2.8.8'
>>> nx.to_numpy_array(G1, dtype=int)
array([[1]])

Trying to apply that to the source code with the below patch
fixed the unit test issue, and if it hasn't changed in version
3.0, then this should continue working onwards:

--- nitime.orig/nitime/viz.py
+++ nitime/nitime/viz.py
@@ -680,7 +680,7 @@

     # Build a 'weighted degree' array obtained by adding the (absolute value)
     # of the weights for all edges pointing to each node:
-    amat = nx.adjacency_matrix(G).todense()  # get a normal array out of it
+    amat = nx.to_numpy_array(G)  # get a normal array out of it
     degarr = abs(amat).sum(0)  # weights are sums across rows

     # Map the degree to the 0-1 range so we can use it for sizing the nodes.

Note that I have not checked the typing actually needed, so
stuck to the default float.

Thank you both for your help!

[effigies]

Excellent! Would you care to submit your patch as a PR?

[emollier]

Here you go. :)

[arokem]

Closed by #208