Get some kind of typing for scipy.special

[person142]

This is a tracking issue for getting some kind of typing (likely very rough) for everything in scipy.special. Things that this entails:

• Type cython_special as Any, since it has no public Python API.
• Type the ufuncs in _ufuncs as best we can
  • Initial PR typing them as Any: MAINT: special: add rudimentary types for _ufuncs extension module #11701
  • See e.g. ENH: add basic typing for np.ufunc numpy/numpy-stubs#44 for an (admittedly not great) initial effort at more detailed types for ufuncs
• Generate stubs for specfun.
  • The interesting thing here will be generating stubs from the .pyf file; I have a local branch with an initial effort
  • Some compromises will need to be made as the "functions" in the modules f2py generates aren't actually functions; e.g. type(sc.specfun.jdzo) is <class 'fortran'>. Since these functions aren't exposed publicly in any way (instead they are just used in _basic.py mostly) we should likely just ignore that distinction.
• Add type stubs for a few more miscellaneous extension modules (_comb etc.)
• Add types for orthogonal.py
• Add types to _basic.py
  • The blocker here will likely be getting something for ArrayLike in the NumPy stubs EDIT: this is included in NumPy 1.20
• Add ignores for the places we import matplotlib/pytest/sympy

My hope is that the process of trying to type special will reveal the big things that need to be solved. Once we have solved them we might be ready to open up a big issue for "adding types to stuff" that will then be beginner friendly.

[rgommers]

With NumPy 1.20.0 I think the things we need (like numpy.typing.ArrayLike) are in place, and we can think about moving type annotations inline.

There's two recent PRs that looked at type annotations, gh-10844 for stats.qmc (tried inline in Python code, failed on Sphinx warnings so far) and gh-13313 for Qhull (inline in Cython code).

The idea of using inline annotations in combination with numpy.typing by using delayed evaluation of annotations via from __future__ import annotations (see #10844 (comment)) looks like the right thing to do.

The current issue is that Sphinx is generating warnings about inline annotations, see #10844 (comment). sphinx.ext.autodoc has a few options to deal with that, like autodoc_typehints = 'none' (Sphinx docs) - but that's not working - probably because of lack of support in numpydoc. Related issues:

• Support type annotations (PEP 484) numpy/numpydoc#196 - numpydoc tracking issue for typing support
• DOC: documentation rendering issue for functions using typing overloads pandas-dev/pandas#37239 (comment) - Pandas does make it work (unclear why, still investigating)
• Support typing scikit-learn/scikit-learn#16705 - scikit-learn tracking issue, with also some discussion of how things show up in IPython

[rgommers]

Type annotations for other submodules are also welcome now - there's a lot to do. Easiest to keep it in stub files for now, gh-13833 is close to finalizing moving them inline.

[vladutmd]

I'm going to start by changing the necessary annotations to ArrayLike in orthogonal.pyi then do basic.pyi

[BvB93]

Recently there was an update to the np.ufunc stubs (see numpy.typing._ufunc) that might be useful as a potential guideline.

Most notably, it introduced a number of type-check-only ufunc subclasses, one for each combination of ufunc.nin and ufunc.nout. In this manner at least some type safety can be guaranteed for each nin/nout combination.

[JPena-code]

Hi. I'm new at contributing to open source and I'm really eager to it, so after reading some of the related issues and PRs (mainly #13833). I would like to try add the typing for the _basic.py module if no one if currently working on it.
Before I start working on it, i wonder if is more suitable to make those typings inline or an stub file?

[lucascolley]

Hi @JPena-code , others my have different thoughts, but I think it makes sense to start with stubs - then you can always move them inline as a follow-up later.

Feel free to ping me if you'd like any help with submitting a PR by the way!

[rgommers]

Almost all existing annotations are in stub files:

$ ls scipy/*/*.pyi
scipy/linalg/_cythonized_array_utils.pyi	scipy/spatial/_qhull.pyi			scipy/special/_test_internal.pyi
scipy/linalg/_decomp_lu_cython.pyi		scipy/spatial/_voronoi.pyi			scipy/special/cython_special.pyi
scipy/linalg/_matfuncs_expm.pyi			scipy/spatial/distance.pyi			scipy/stats/_qmc_cy.pyi
scipy/spatial/_ckdtree.pyi			scipy/special/_orthogonal.pyi			scipy/stats/_sobol.pyi

so adding more there seems like the right default for now.

[JPena-code]

Great, thanks for the help @lucascolley, I will try to add the typing in a stub file