Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MAINT: remove numpy.array_api module #25911

Merged
merged 4 commits into from
Mar 3, 2024

Conversation

rgommers
Copy link
Member

@rgommers rgommers commented Mar 1, 2024

This submodule was added in NumPy 1.22 under an experimental label. After gaining experience with it, we decided that it was quite useful for testing code in downstream libraries and from end users written against the array API standard, but not for production code. Now that we have array API standard support in the main namespace, we therefore decided to remove this submodule - as documented in NEP 56.

The code from numpy.array_api was moved to the array-api-strict package (see https://github.com/data-apis/array-api-strict) and further improved there. Users are advised to start using that standalone package for testing their code if they want to check that it works not only with numpy but with other libraries that support the array API standard.

This commit also updates the entrypoint (used by for example hypothesis) to point to the main namespace. Users of this entrypoint to obtain NumPy's compliant namespace should not be affected by this change.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've linked to this document in quite a few places. Obviously with numpy 2.0, most of what was here is no longer true. Is it worth keeping the contents here around somewhere (maybe in the array-api-strict docs)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will remain available at https://numpy.org/doc/1.26/reference/array_api.html#table-of-differences-between-numpy-array-api-and-numpy. How about I add that link to this page?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a link to NEP 47, and cross-linked NEP 47 and 56.

I'll also have to follow up on the mailing list about changing the status of NEPs (superseded, accepted).

main ``numpy`` namespace would not need to change in any way to be
spec-compatible for these.
NumPy installs an `entry point <https://packaging.python.org/en/latest/specifications/entry-points/>`__
that can be used for discovery purposes::
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I never noticed that this was actually added to the standard. I should probably add it to array-api-compat and array-api-strict.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's marked as optional and not in the API docs, since it isn't actually an API but a packaging thing. Maybe we should improve its visibility though?

@asmeurer
Copy link
Member

asmeurer commented Mar 1, 2024

Obviously +1 to the overall change here.

I don't know how NEPs work, but should the NEP that proposed numpy.array_api be updated?

@rgommers
Copy link
Member Author

rgommers commented Mar 3, 2024

I don't know how NEPs work, but should the NEP that proposed numpy.array_api be updated?

Good point, I'll mark it as superseded and add a little note.

@mattip
Copy link
Member

mattip commented Mar 3, 2024

The macos failure seems relevant

from the log
>       xp = ep.load()

ep         = EntryPoint(name='numpy', value='numpy.array_api', group='array_api')
eps        = {'array_api': [EntryPoint(name='numpy', value='numpy.array_api', group='array_api')], 'babel.checkers': [EntryPoint(na...hinx_theme', group='sphinx.html_themes'), EntryPoint(name='alabaster', value='alabaster', group='sphinx.html_themes')]}
numpy_in_sitepackages = False
xp_eps     = [EntryPoint(name='numpy', value='numpy.array_api', group='array_api')]

numpy/tests/test_public_api.py:482: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/Users/runner/miniconda3/envs/numpy-dev/lib/python3.11/importlib/metadata/__init__.py:202: in load
    module = import_module(match.group('module'))
        match      = <re.Match object; span=(0, 15), match='numpy.array_api'>
        self       = EntryPoint(name='numpy', value='numpy.array_api', group='array_api')
/Users/runner/miniconda3/envs/numpy-dev/lib/python3.11/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
        level      = 0
        name       = 'numpy.array_api'
        package    = None
<frozen importlib._bootstrap>:1204: in _gcd_import
    ???
        level      = 0
        name       = 'numpy.array_api'
        package    = None
<frozen importlib._bootstrap>:1176: in _find_and_load
    ???
        import_    = <function _gcd_import at 0x1031cbd80>
        module     = <object object at 0x103200050>
        name       = 'numpy.array_api'
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

name = 'numpy.array_api', import_ = <function _gcd_import at 0x1031cbd80>

>   ???
E   ModuleNotFoundError: No module named 'numpy.array_api'

child      = 'array_api'
import_    = <function _gcd_import at 0x1031cbd80>
name       = 'numpy.array_api'
parent     = 'numpy'
parent_module = <module 'numpy' from '/Users/runner/work/numpy/numpy/build-install/usr/lib/python3.11/site-packages/numpy/__init__.py'>
parent_spec = ModuleSpec(name='numpy', loader=<_frozen_importlib_external.SourceFileLoader object at 0x104735110>, origin='/Users/ru...y', submodule_search_locations=['/Users/runner/work/numpy/numpy/build-install/usr/lib/python3.11/site-packages/numpy'])
path       = ['/Users/runner/work/numpy/numpy/build-install/usr/lib/python3.11/site-packages/numpy']
spec       = None

<frozen importlib._bootstrap>:1140: ModuleNotFoundError
=========================== short test summary info ============================
FAILED numpy/tests/test_public_api.py::test_array_api_entry_point - ModuleNotFoundError: No module named 'numpy.array_api'

@rgommers
Copy link
Member Author

rgommers commented Mar 3, 2024

For the test failure on macOS, what is happening is that the entrypoint is not installed by spin (it doesn't know how to do that), but because the job uses environment.yml there is another numpy package (1.26.4) installed which does install the entrypoint, pointing to the now-removed submodule. This is kinda an invalid situation, so we just need to recognize it and not try to load the entrypoint in that case.

This submodule was added in NumPy 1.22 under an experimental label.
After gaining experience with it, we decided that it was quite
useful for testing code in downstream libraries and from end users
written against the array API standard, but not for production code.
Now that we have array API standard support in the main namespace,
we therefore decided to remove this submodule - as documented in
NEP 56

The code from `numpy.array_api` was moved to the `array-api-strict`
package (see https://github.com/data-apis/array-api-strict) and
further improved there. Users are advised to start using that standalone
package for testing their code if they want to check that it works
not only with `numpy` but with other libraries that support the
array API standard.

This commit also updates the entrypoint (used by for example
`hypothesis`) to point to the main namespace. Users of this entrypoint
to obtain NumPy's compliant namespace should not be affected by this
change.

[skip cirrus]
Copy link
Member

@mattip mattip left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, once CI is passing.

# This isn't a problem except for when running tests with `spin` or an
# in-place build.
return

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool.

@rgommers
Copy link
Member Author

rgommers commented Mar 3, 2024

CI is happy aside from an unrelated connection issue on the FreeBSD job. PyPy is still running but also happy so far and passed on the last run already. Let's call it good. Thanks for the reviews Matti and Aaron!

@rgommers rgommers merged commit 8d9218d into numpy:main Mar 3, 2024
60 of 63 checks passed
@rgommers rgommers deleted the remove-arrayapi-module branch March 3, 2024 16:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants