MAINT: Remove unsafe unions and ABCs from return-annotations #18915
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport of #18885.
Per the title, this PR removes unsafe unions and abstract baseclasses from the return annotations,
e.g. functions that currently have one of the following patterns.
The Problem
The problem with returning a
Union
(or, almost equivalently, an abstract-ish baseclass such asgeneric
)is that any and all operations performed on a union must be compatible with all of its members.
For example, operations that are exclusive to either
np.float64
andnp.ndarray
are thus not allowedto be executed by
np.float64 | np.ndarray
, unless the union is narrowed down via an explicitisinstance
check a priori.
While returning a union thus adds some a form of simplicity to the annotations (as we don't have to
distinguish between 0D and ND array-likes), the
Union
type is simply not suited for what we're tryingto describe here (xref python/mypy#1693). This would be a different story
for a hypothetical
UnsafeUnion
type, one where operations must be compatible with any memberof the union. Such type does not exist though.
The Solution
The solutions implemented herein fall in either one of the following two categories:
Any
. This is a simple, but non-ideal solution, as it removes any typesafety for objects returned by aforementioned functions. Nevertheless, there is a group of functions
that still needs to-be updated for dtype-support anyway (e.g. those in
np.core.fromnumeric
),so setting their return to
Any
is by no means the worst thing that can happen.A second group of functions is those where the use of
Any
is simply a necasity, as the output type is.For example, determined by the value of string literals (see
np.einsum
). As a silver lining: the einsumproblem in particular does seem like it can be resolved with relative ease via a future mypy plugin.
for the unsafe-union issue; it has been applied to the more recently annotated modules such as
np.lib.ufunclike
.There is however one important caveat here: as we currently lack shape-support (Typing support for shapes #16544) it is
currently impossible to distinguish between 0D and ND ndarrays, and thus we are unable to describe
the 0D-to-scalar casting that numpy aggressively performs on 0D arrays. While this should change
once PEP 646 is live, in the mean time users will have to settle for a
typing.cast
call or a# type: ignore
comment if it is known in advance that 0D-to-scalar cast will be performed.