BUG: Sparse matrix todense doesn't roll (np.roll)

[abaxi]

If I make a sparse matrix, pick out one row from it, call to todense() on the row, and finally call np.roll(row, 1), then the row isn't rolled.

If I define row_np = np.array(row), and then call np.roll(row_np, 1), then it rolls.

Is this behaviour how it is intended (and I'm missing some underlying logic), or is it a bug?

Should this be posted in the scipy issues instead?

Thank you.

Reproducing code example:

import numpy as np
from scipy.sparse import csr_matrix

nrows = 3
ncols = 5
sparse_data = [1,2,3]
sparse_i = [0,1,2]
sparse_j = [0,2,4]
Y = csr_matrix((sparse_data, (sparse_i, sparse_j)), shape=(nrows, ncols))

y_0 = Y[0].todense()
print "y_0:", y_0
print "y_0 rolled by 1:", np.roll(y_0, 1)
y_0 = np.array(y_0)
print "np.array(y_0) rolled by 1:", np.roll(y_0, 1)

Outputs:

y_0: [[1 0 0 0 0]]
y_0 rolled by 1: [[1 0 0 0 0]]
np.array(y_0) rolled by 1: [[0 1 0 0 0]]

Numpy/Python version information:

Output from 'import sys, numpy; print(numpy.version, sys.version)'

('1.15.1', '2.7.12 (default, Nov 12 2018, 14:36:49) \n[GCC 5.4.0 20160609]')

import sys, scipy; print(scipy.version, sys.version)
('0.17.0', '2.7.12 (default, Nov 12 2018, 14:36:49) \n[GCC 5.4.0 20160609]')

[WarrenWeckesser]

The problem has nothing to do with SciPy's sparse matrices. The problem is apparently a bug (or perhaps just a "gotcha") in how np.roll handles matrix arguments when axis=None. Y[0].todense() returns a matrix (not ndarray) with shape (1, 5), and applying roll to this matrix exposes the unexpected behavior.

For example, in the following, np.roll(a, 1) works as expected:

In [74]: a = np.array([[0, 1, 2, 3, 4]])

In [75]: np.roll(a, 1)
Out[75]: array([[4, 0, 1, 2, 3]])

Given that, it is surprising that np.roll(m, 1) does not produce a similar result in the following:

In [76]: m = np.matrix([[0, 1, 2, 3, 4]])

In [77]: np.roll(m, 1)
Out[77]: matrix([[0, 1, 2, 3, 4]])

The docstring for roll says that if axis=None (the default), "the array is flattened before shifting, after which the original shape is restored", and here's what roll returns when axis is None:

numpy/numpy/core/numeric.py

Line 1213 in 8877660

return roll(a.ravel(), shift, 0).reshape(a.shape)

The problem is that the ravel() method does not flatten a matrix. matrix objects can't really be "flattened"--at least not while remaining matrix objects. They insist on remaining 2-d:

In [78]: m.ravel()
Out[78]: matrix([[0, 1, 2, 3, 4]])

In [79]: m.flatten()
Out[79]: matrix([[0, 1, 2, 3, 4]])

so np.roll(m, 1) ends up rolling the 2-d matrix along axis 0, which has length 1, so there is no effect.

In this example, a workaround is to use axis=1:

In [80]: np.roll(m, 1, axis=1)
Out[80]: matrix([[4, 0, 1, 2, 3]])

Alternatively, avoid the matrix class altogether by using the toarray() method of SciPy's sparse matrix instead of the todense() method:

In [87]: y_0 = Y[0].toarray()  # Use `.toarray()` instead of `.todense()`.

In [88]: y_0
Out[88]: array([[1, 0, 0, 0, 0]])

In [89]: np.roll(y_0, 1)
Out[89]: array([[0, 1, 0, 0, 0]]

The matrix class is destined for deprecation, so it might not be worthwhile spending more time on this quirk.