BUG: utils from scipy.sparse.sputils are not available from scipy.sparse module

[glemaitre]

Describe your issue.

It seems that the deprecation warning of the scipy.sparse.sputils is incorrect. The utilities are not available from the scipy.sparse module.

This might be on purpose but then the deprecation warning is misleading.

Reproducing Code Example

In [2]: from scipy.sparse.sputils import get_index_dtype
<ipython-input-2-8d76e9dbe0fa>:1: DeprecationWarning: Please use `get_index_dtype` from the `scipy.sparse` namespace, the `scipy.sparse.sputils` namespace is deprecated.
  from scipy.sparse.sputils import get_index_dtype

In [3]: from scipy.sparse import get_index_dtype
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[3], line 1
----> 1 from scipy.sparse import get_index_dtype

ImportError: cannot import name 'get_index_dtype' from 'scipy.sparse' (/home/glemaitre/mambaforge/envs/dev/lib/python3.10/site-packages/scipy/sparse/__init__.py)

Error message

---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
Cell In[3], line 1
----> 1 from scipy.sparse import get_index_dtype

ImportError: cannot import name 'get_index_dtype' from 'scipy.sparse' (/home/glemaitre/mambaforge/envs/dev/lib/python3.10/site-packages/scipy/sparse/__init__.py)

SciPy/NumPy/Python version and system information

In [4]: import scipy

In [5]: scipy.__version__
Out[5]: '1.11.2'

[j-bowhay]

This will be fixed by #19226 as part of #18279. The question is: is get_index_dtype meant to be public?

[glemaitre]

Thanks @j-bowhay.

The question is: is get_index_dtype meant to be public?

Some context that could be useful for the discussion.

We are currently working compatibility sparse matrices/arrays in scikit-learn. We stumbled into issues linked to the discussion here: #16774: sparse matrices and sparse arrays will differ sometimes regarding the data types of the indices.

The issue on our side is that we can have some format conversion, namely dia_arrays.tocsr(), leading to 64-bits indices dtype. It becomes an issue when those matrices are provided to low-level implementation where the type was fixed to 32-bits.

While it makes little sense asking for a backward compatibility while it has been decided to not downcast, we would still need to do so in some specific case in scikit-learn to make sure that we don't break the code base. Having this utility can then help use to post-downcast the indices.

This said, we could also vendor this helper since the implementation is quite small and straightforward.

[j-bowhay]

@perimosocordiae do you have any opinion on whether this function was meant to be public (ie. should we expose it in scipy.sparse)? A search suggests that there isn't much usage so my opinion is probably no, sklearn should vendor the helper.

[perimosocordiae]

I'd prefer not to make this public.