Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot create sparse matrix from size_t indexes #9437

Closed
david-cortes opened this issue Nov 4, 2018 · 1 comment · Fixed by #9438
Closed

Cannot create sparse matrix from size_t indexes #9437

david-cortes opened this issue Nov 4, 2018 · 1 comment · Fixed by #9438
Labels
defect A clear bug or issue that prevents SciPy from being installed or used as expected scipy.sparse
Milestone

Comments

@david-cortes
Copy link
Contributor

david-cortes commented Nov 4, 2018

If I try to create a sparse matrix in COOrdinate format with row and column indexes in size_t type, coo_matrix will throw an error as it seems it tries to cast size_t to float64.

The following code illustrates the problem:

import numpy as np
import ctypes
from scipy.sparse import coo_matrix

nrow = 10**4
ncol = 10**5
nnz = 10**6
np.random.seed(1)
row_id = np.random.randint(nrow, size=nnz).astype(ctypes.c_size_t)
col_id = np.random.randint(ncol, size=nnz).astype(ctypes.c_size_t)
vals = np.random.normal(size=nnz)

sp_matrix = coo_matrix((vals, (row_id, col_id)))

Throws:

Traceback (most recent call last):

  File "<ipython-input-1-5bbb7c9f9698>", line 13, in <module>
    sp_matrix = coo_matrix((vals, (row_id, col_id)))

  File "/home/david/anaconda3/lib/python3.6/site-packages/scipy/sparse/coo.py", line 150, in __init__
    self._shape = check_shape((M, N))

  File "/home/david/anaconda3/lib/python3.6/site-packages/scipy/sparse/sputils.py", line 281, in check_shape
    new_shape = tuple(operator.index(arg) for arg in args)

  File "/home/david/anaconda3/lib/python3.6/site-packages/scipy/sparse/sputils.py", line 281, in <genexpr>
    new_shape = tuple(operator.index(arg) for arg in args)

TypeError: 'numpy.float64' object cannot be interpreted as an integer

Adding the shape will however allow to construct this matrix:

coo_matrix((vals, (row_id, col_id)), shape=(nrow, ncol))
<10000x100000 sparse matrix of type '<class 'numpy.float64'>'
	with 1000000 stored elements in COOrdinate format>

The problem is not present when the indexes are of int or long type:

coo_matrix((vals, (row_id.astype(ctypes.c_long), col_id.astype(ctypes.c_long))))
<10000x100000 sparse matrix of type '<class 'numpy.float64'>'
	with 1000000 stored elements in COOrdinate format>
@WarrenWeckesser WarrenWeckesser added defect A clear bug or issue that prevents SciPy from being installed or used as expected scipy.sparse labels Nov 4, 2018
@pv
Copy link
Member

pv commented Nov 4, 2018

The index needs to be signed integer, size_t is unsigned.

The error message is perhaps confusing, but it's because in casting float64 is the next common type uint64 and signed integers can be cast to.

EDIT: ok, the shape check should not fail like this nevertheless

@rgommers rgommers added this to the 1.3.0 milestone Feb 28, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
defect A clear bug or issue that prevents SciPy from being installed or used as expected scipy.sparse
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants