decorator fails on NDArray[Any] #69
Comments
|
Great catch, Jasha. I shamefully missed that in the NumPy docos – but should have known better. Everything is subscriptable by I'll resolve this straight-away for the global benefit of data scientists everywhere. Thanks again for the heads-up! Double thumbs up to your avatar equipped with stylish coke-bottle glasses, too. 👍 🤓 👍 |
|
Oh, vile Gods below. My younger, smarter, more virile self even authored a #FIXME: Reduce both the unsubscripted "numpy.typing.NDArray" class *AND* the
#uselessly subscripted "numpy.typing.NDArray[typing.Any]" type hint to the
#"numpy.ndarray" class.It's not a good look. 🤕 Relatedly, I'll go ahead and also add support for the unsubscripted |
leycec
added a commit
that referenced
this issue
Nov 16, 2021
This commit adds support for **untyped NumPy array type hints** (i.e., the unsubscripted `numpy.typing.NDArray` and subscripted `numpy.typing.NDArray[typing.Any]` type hints), resolving issue #69 kindly submitted by @Jasha10, the stylish boy wonder dual-wielding the double thumbs-up and coke-bottle glasses that signify elementary genius. Specifically, this commit now detects and reduces untyped NumPy array type hints to the semantically equivalent `numpy.ndarray` class. (*Plums lobbed with aplomb!*)
|
Resolved by 2dbe536. It. Is. Done. Thanks so much for prodding me to get it done too, Jasha. If you're willing to pretend this never happened, I'm willing to pretend I still have hair. I'd like to kick out a new stable release with this fix in a week or two – but it might be longer. You just never know with @leycec, right? In the meanwhile, all of the following type hints should be semantically equivalent to from beartype import beartype
import numpy as np
import numpy.typing as npt
from typing import Any
# Substitute "Any" -> "np.generic" to make the bear stop growling.
@beartype
def foo(bar: npt.NDArray[np.generic]) -> Any:
...
# Substitute "npt.NDArray[Any]" -> "np.ndarray". (This is so lame.)
@beartype
def foo(bar: np.ndarray) -> Any:
...I'd probably go with |
|
Haha, thanks @leycec! |
leycec
added a commit
that referenced
this issue
Feb 9, 2022
This release titillates with scintillating support for **[PEP 557 -- Data Classes][PEP 557]**, **[PEP 570 -- Python Positional-Only Parameters][PEP 570]**, and **[PEP 604 -- Allow writing union types as X | Y][PEP 604]**. This release resolves a bone-crushing **30 issues** (mostly shameless dupes of one another, admittedly) and merges **3 pull requests.** World-girdling changes include: ## Compatibility Added * **[PEP 557 -- Data Classes][PEP 557].** `@beartype` now supports **dataclasses** (i.e., types decorated by the standard `@dataclasses.dataclass` decorator), resolving issue #56 kindly submitted by @JulesGM (Jules Gagnon-Marchand) the Big Brain NLP researcher. Specifically, `@beartype` now transparently type-checks: * **Dataclass-specific initialization-only instance variable type hints** (i.e., `dataclasses.InitVar[...]`). * The implicit `__init__()` method generated by `@dataclass` for dataclasses through a clever one-liner employed by @antonagestam (Anton Agestam) the ageless Swede that I stan for. * **[PEP 570 -- Python Positional-Only Parameters][PEP 570].** `@beartype` now supports positional-only arguments and no one cares. Given the triviality, the rear view mirror of regret suggests we kinda should've implemented this sooner. Better late than never, best @beartype friends for life (BBFFL). * **[PEP 604 -- Allow writing union types as X | Y][PEP 604].** `@beartype` now supports new-style set unions (e.g., `int | float`), resolving issue #71 kindly submitted by pro typing aficionado Derek Wan (@dycw). Thanks to Derek for the helpful heads up that @beartype was headed straight for typing disaster under Python ≥ 3.10. Since we dodged another bullet there, this must mean we have now activated bullet time. Goooooo, slomo! ## Compatibility Improved * **[PEP 484 -- Type Hints][PEP 484],** including: * **`typing.{Binary,Text,}IO[...]` deep type-checking.** `@beartype` now deeply type-checks subscripted `typing.{Binary,Text,}IO[...]` type hints, resolving issue #75 kindly submitted by Niklas "If I had a nickel for every lass..." Rosenstein. Notably: * Since the `typing.BinaryIO` protocol and its `typing.IO` superclass share the exact same API, the `typing.BinaryIO` protocol is lamentably useless for *all* practical purposes. This protocol *cannot* be leveraged to detect binary file handles. Can binary file handles be detected at runtime then? Yes, we can! A binary file handle is any object satisfying the `typing.IO` protocol but *not* the `typing.TextIO` protocol. To implement this distinction, `@beartype` necessarily invented a novel form of type-checking and a new variant of type elision: **anti-structural subtyping.** Whereas structural subtyping checks that one class matches the API of another class (referred to as a "protocol"), anti-structural subtyping checks that one class does *not* match the API of another class (referred to as an "anti-protocol"). `@beartype` public exposes this functionality via the new `beartype.vale.IsInstance[...]` validator, enabling *anyone* to trivially perform anti-structural subtyping. In this case, `@beartype` internally reduces all useless `typing.BinaryIO` type hints to substantially more useful `typing.Annotated[typing.IO, ~beartype.vale.IsInstance[typing.TextIO]]` type hints. * **Unsubscripted NumPy type hints.** `@beartype` now supports **untyped NumPy array type hints** (i.e., the unsubscripted `numpy.typing.NDArray` and subscripted `numpy.typing.NDArray[typing.Any]` type hints), resolving issue #69 kindly submitted by @Jasha10, the stylish boy wonder dual-wielding the double thumbs-up and coke-bottle glasses that signify elementary genius. Specifically, this commit now detects and reduces these hints to the equivalent `numpy.ndarray` type. * **Mypy ≥ 0.920.** `@beartype` now squelches ignorable mypy complaints first introduced by mypy 0.920, including: * **Explicit reexport errors.** `beartype` now squelches implicit reexport complaints from mypy with respect to public attributes published by the `beartype.cave` subpackage, resolving issue #57 kindly reopened by Göteborg melodic death metal protégé and brightest academic luminary @antonagestam. This subpackage is now compatible with both the `--no-implicit-reexport` mypy CLI option and equivalent `no_implicit_reexport = True` configuration setting in `.mypy.ini`. * **Version-dependent errors.** Previously, mypy permitted imports against standard library modules introduced in newer CPython versions to be squelched with the usual ``"# type: ignore[attr-defined]"`` pragma. Since mypy now ignores these pragmas, `@beartype` now silences its complaints through... *unconventional* means. A bear do wut a bear gotta do. ## Features Added * **Compatibility API.** `beartype` now publishes a new `beartype.typing` API as a `typing` compatibility layer improving forward compatibility with future Python releases, resolving issue #81 kindly submitted by the honorable @qiujiangkun (Qiu Jiangkun). Consider resolving [PEP 585][PEP 585] deprecations by importing from our new `beartype.typing` API rather than the standard `typing` API. A battery of new unit tests ensure conformance: * Between `beartype.typing` and `typing` across all Python versions. * With mypy when importing from `beartype.typing`. * **Configuration API** (i.e., public attributes of the `beartype` package enabling end users to configure the `@beartype` decorator, including configuring alternative type-checking strategies *other* than constant-time runtime type-checking). Specifically, `beartype` now publishes: * `beartype.BeartypeStrategy`, an enumeration of all type-checking strategies to *eventually* be fully supported by future beartype releases – including: * `BeartypeStrategy.O0`, disabling type-checking for a callable by reducing `@beartype` to the identity decorator for that callable. Although currently useless, this strategy will usefully allow end users to selectively prevent callables from being type-checked by our as-yet-unimplemented import hook. When implemented, that hook will type-check *all* callables in a given package by default. Some means is needed to prevent that from happening for select callables. This is that means. * `BeartypeStrategy.O1`, our default `O(1)` constant-time strategy type-checking a single randomly selected item of a container that you currently enjoy. Since this is the default, this strategy need *not* be explicitly configured. Of course, you're going to do that anyway, aren't you? `</sigh>` * `BeartypeStrategy.Ologn`, a new `O(lgn)` logarithmic strategy type-checking a randomly selected number of items `j` of a container `obj` such that `j = len(obj)`. This strategy is **currently unimplemented** (but will be implemented by a future beartype release). * `BeartypeStrategy.On`, a new `O(n)` linear strategy deterministically type-checking *all* items of a container. This strategy is **currently unimplemented** (but will be implemented by a future beartype release). * `beartype.BeartypeConf`, a simple dataclass encapsulating all flags, options, settings, and other metadata configuring the current decoration of the decorated callable or class. For efficiency, this dataclass internally self-caches itself (i.e., `BeartypeConf(*args, **kwargs) is BeartypeConf(*args, **kwargs)`). The `__init__()` method of this dataclass currently accepts these optional parameters: * An `is_debug` boolean instance variable. When enabled, `@beartype` emits debugging information for the decorated callable – including the code for the wrapper function dynamically generated by `@beartype` that type-checks that callable. * A `strategy` instance variable whose value must be a `BeartypeStrategy` enumeration member. This is how you notify `@beartype` of which strategy to apply to each callable. * **Wrapper function debuggability.** Enabling the `is_debug` parameter to the `BeartypeConf.__init__` method significantly improves the debuggability of type-checking wrapper functions generated by `@beartype`. This configuration option is entirely thanks to @posita the positive Numenorean, who pined longingly for debuggable wrapper functions and now receives proportionately. Praise be to @posita! He makes bears better. Specifically, enabling this option enables developer-friendly logic like: * Pretty-printing to stdout (standard output) the definitions of those functions, including line number prefixes for readability. * Enabling those functions to be debugged. Thanks to a phenomenal pull request by the dynamic dual threat that is @posita **+** @TeamSpen210, `@beartype` now conditionally caches the bodies of type-checking wrapper functions with the standard (albeit poorly documented) `linecache` module. Thanks so much! Bear Clan 2022!!! * Suffixing the declarations of `@beartype`-specific hidden private "special" parameters passed to those functions with comments embedding their human-readable representations. Safely generating these comments consumes non-trivial wall clock at decoration time and is thus conditionally enabled for external callers requesting `@beartype` debugging. For example, note the `"# is"`-prefixed comments in the following signature of a `@beartype`-generated wrapper function for an asynchronous callable with signature `async def control_the_car(said_the: Union[str, int], biggest_greenest_bat: Union[str, float]) -> Union[str, float]:` ``` python (line 0001) async def control_the_car( (line 0002) *args, (line 0003) __beartype_func=__beartype_func, # is <function test_decor_async_coroutine.<locals>.control_the_car at 0x7> (line 0004) __beartype_raise_exception=__beartype_raise_exception, # is <function raise_pep_call_exception at 0x7fa13d> (line 0005) __beartype_object_140328307018000=__beartype_object_140328307018000, # is (<class 'int'>, <class 'str'>) (line 0006) __beartype_object_140328306652816=__beartype_object_140328306652816, # is (<class 'float'>, <class 'str'>) (line 0007) **kwargs (line 0008) ): ``` * **Decorator modality.** `@beartype` now supports two orthogonal modes of operation: * **Decoration mode** (i.e., the standard mode where `@beartype` directly decorates a callable *without* being passed parameters). In this mode, `@beartype` reverts to the default configuration of constant-time runtime type-checking and *no* debugging behaviour. * **Configuration mode** (i.e., the new mode where `@beartype` is called as a function passed a `BeartypeConf` object via the keyword-only `conf` parameter). In this mode, `@beartype` efficiently creates, caches, and returns a memoized decorator encapsulating the passed configuration: e.g., ``` python from beartype import beartype, BeartypeConf, BeartypeStrategy @beartype(conf=BeartypeConf(strategy=BeartypeStrategy.On)) def muh_func(list_checked_in_linear_time: list[int]) -> int: return len(list_checked_in_linear_time) ``` * Specifically, this commit extricates our core `@beartype` decorator into a new private `beartype._decor._core` submodule in preparation for subsequently memoizing closures encapsulating that decorator returned by invocations of the form `@beartype.beartype(conf=BeartypeConf(...))` * **Declarative instance validator.** `beartype` now publishes a new `beartype.vale.IsInstance[...]` validator enforcing instancing of one or more classes, generalizing **isinstanceable type hints** (i.e., normal pure-Python or C-based classes that can be passed as the second parameter to the ``isinstance()`` builtin). Unlike standard isinstanceable type hints, `beartype.vale.IsInstance[...]` supports various set theoretic operators. Critically, this includes negation. Instance validators prefixed by the negation operator `~` match all objects that are *not* instances of the classes subscripting those validators. Wait. Wait just a hot minute there. Doesn't a typing.Annotated_ type hint necessarily match instances of the class subscripting that type hint? Yup. This means type hints of the form `typing.Annotated[{superclass}, ~IsInstance[{subclass}]` match all instances of a superclass that are *not* also instances of a subclass. And... pretty sure we just invented type hint arithmetic right there. That sounded intellectual and thus boring. Yet, the disturbing fact that Python booleans are integers <sup>yup</sup> while Python strings are infinitely recursive sequences of strings <sup>yup</sup> means that type hint arithmetic can save your codebase from Guido's younger self. Consider this instance validator matching only non-boolean integers, which *cannot* be expressed with any isinstanceable type hint (e.g., ``int``) or other combination of standard off-the-shelf type hints (e.g., unions): `Annotated[int, ~IsInstance[bool]]`. ← *bruh* * **Functional API.** `beartype` now publishes a new public `beartype.abby` subpackage enabling users to type-check *anything* *anytime* against *any* PEP-compliant type hints, resolving feature request #79 kindly submitted by (*...wait for it*) typing Kung Fu master @qiujiangkun (Qiu Jiangkun). This subpackage is largely thanks to @qiujiangkuni, whose impeccable code snippets drive our initial implementation. This subpackage provides these utility functions: * `beartype.abby.is_bearable()`, strictly returning a boolean signifying whether the passed arbitrary object satisfies the passed type hint or not (e.g., `is_bearable(['the', 'centre', 'cannot', 'hold;'], list[int]) is False`). * `beartype.abby.die_if_unbearable()`, raising the new `beartype.roar.BeartypeAbbyHintViolation` exception when the passed arbitrary object violates the passed type hint. ## Features Improved * **Exception message granularity,** including exceptions raised for: * **Disordered builtin decorators.** `@beartype` now raises instructive exceptions when decorating an uncallable descriptor created by a builtin decorator (i.e., `@property`, `@classmethod`, `@staticmethod`) due to the caller incorrectly ordering `@beartype` above rather than below that decorator, resolving issue #80 kindly submitted by typing academician @qiujiangkun (Qiu Jiangkun). Specifically, `@beartype` now raises human-readable exceptions suffixed by examples instructing callers to reverse decoration ordering. * **Beartype validators.** `@beartype` now appends a detailed pretty-printed diagnosis of how any object either satisfies or fails to satisfy any beartype validator to exception messages raised by high-level validators synthesized from lower-level validators (e.g., via overloaded set theoretic operators like `|`, `&`, and `~`), resolving issue #72 kindly submitted by the unwreckable type-hinting guru Derek Wan (@dycw). This diagnostic trivializes validation failures in non-trivial use cases involving multiple nested conjunctions, disjunctions, and/or negations. ## Features Optimized * **`@beartype` call-time performance.** `@beartype` now generates faster type-checking wrapper functions with a vast and undocumented arsenal of absolutely "legal" weaponry, including: * **`typing.{Generic,Protocol}` deduplication.** `@beartype` now microoptimizes away redundant `isinstance()` checks in wrapper functions checking `@beartype`-decorated callables annotated by **PEP 484-compliant subgenerics or PEP 585-compliant subprotocols** (i.e., user-defined classes subclassing user-defined classes subclassing `typing.{Generic, Protocol}`), resolving issue #76 kindly submitted by @posita the positive numerics QA guru and restoring the third-party `numerary` package to its glory. Our generics workflow has been refactored from the ground-up to stop behaving insane. `@beartype` now performs an inner breadth-first search (BFS) across generic pseudo-superclasses in its existing outer BFS that generates type-checking code. When you're nesting a BFS-in-a-BFS, your code went full-send. There's no going back from that. * **Worst-case nested data structures.** `@beartype` now resolves a performance regression in type-checking wrapper functions passed worst-case nested data structures violating PEP-compliant type hints, resolving issue #91 kindly submitted by Cuban type-checking revolutionary @mvaled (Manuel Vázquez Acosta). Specifically, this commit safeguards our low-level `represent_object()` function stringifying objects embedded in exception messages describing type-checking violations against worst-case behaviour. A new unit test shieldwalls against further performance regressions. All our gratitude to @mvaled for unveiling the darkness in the bear's heart. * **`@beartype` decoration-time performance.** The `@beartype` decorator has been restored to its prior speed, resolving performance regressions present throughout our [0.8.0, 0.10.0) release cycles. Significant decoration-time optimizations include: * **Code objects.** `@beartype` now directly accesses the code object underlying the possibly unwrapped callable being decorated via a temporary cache rather than indirectly accessing that code object by repeatedly (and expensively) unwrapping that callable, dramatically optimizing low-level utility functions operating on code objects. * **Exception messages.** `@beartype` now defers calling expensive exception handling-specific functions until an exception is raised, dramatically restoring our decoration-time performance to the pre-0.8.0 era – which isn't that great, honestly. But we'll take anything. Substantial optimizations remain, but we are dog-tired. Moreover, DQXIS:EofaEA (...that's some catchy name right there) ain't gonna play itself – *OR IS IT!?!* Cue creepy AI. * **Fixed lists.** `@beartype` now internally lelaxes inapplicable safety measures previously imposed by our internal `FixedList` container type. Notably, this type previously detected erroneous attempts to extend the length of a fixed list by subversively assigning a slice of that fixed list to a container whose length differs from that of that slice. While advisable in theory, `@beartype` *never* actually sliced any fixed list -- let alone used such a slice as the left-hand side (LHS) of an assignment. Disabling this detection measurably improves the efficiency of fixed lists across the codebase -- which is, after all, the entire raison d'etre for fixed lists in the first place. `</shaking_my_head>` * **Parameter introspection.** `@beartype` now introspects callable signatures using a homegrown lightweight parameter parsing API. `@beartype` previously introspected signatures using the standard heavyweight `inspect` module, which proved... *inadvisable.* All references to that module have been removed from timing-critical code paths. All remaining references reside only in timing-agnostic code paths (e.g., raising human-readable exceptions for beartype validators defined as anonymous lambda functions). * **`@beartype` importation-time performance.** The `beartype` package now avoids unconditionally importing optional first- and third-party subpackages, improving the efficiency of the initial ``from beartype import beartype`` statement in particular. `beartype` now intentionally defers these imports from global module scope to the local callable scope that requires them. A new functional test guarantees this to be the case. ## Features Deprecated * **Badly named exception classes,** to be removed in `beartype` 0.1.0. This includes: * `beartype.roar.BeartypeCallHintPepException`, deprecated by `beartype.roar.BeartypeCallHintViolation`. * `beartype.roar.BeartypeCallHintPepParamException`, deprecated by `beartype.roar.BeartypeCallHintParamViolation`. * `beartype.roar.BeartypeCallHintPepReturnException`, deprecated by `beartype.roar.BeartypeCallHintReturnViolation`. ## Documentation Revised * The *Frequently Asked Questions (FAQ)* section of our front-facing `README.rst` documentation now sports a medley of new entries, including instructions on: * **Boto3 integration,** enabling end users to type-check runtime types dynamically fabricated by Boto3 (i.e., the official Amazon Web Services (AWS) Software Development Kit (SDK) for Python), resolving issue #68 kindly submitted by Paul Hutchings (@paulhutchings) – the supremely skilled sloth rockin' big shades and ever bigger enthusiasm for well-typed Python web apps. Relatedly, the @beartype organization now officially hosts [`bearboto3`, Boto3 @beartype bindings by (*wait for it*) @paulhutchings](https: //github.com/beartype/bearboto3). * **Mock type type-checking,** resolving issue #92 kindly submitted by @Masoudas (Masoud Aghamohamadian-Sharbaf – wish I had an awesome name like that). Gratuitous shoutouts to @TeamSpen210 for the quick save with a ludicrous two-liner solving everything. [PEP 484]: https://www.python.org/dev/peps/pep-0484 [PEP 557]: https://www.python.org/dev/peps/pep-0557 [PEP 570]: https://www.python.org/dev/peps/pep-0570 [PEP 585]: https://www.python.org/dev/peps/pep-0585 [PEP 604]: https://www.python.org/dev/peps/pep-0604 The hype train is now boarding. All aboooooooard! (*Classless masterless masterclass!*)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
In the numpy docs, there is an example showing use of
npt.NDArray[Any]to type-hint an array with unspecified data type.Bear type does not seem to support this.
Reproducing the issue:
At the console:
The text was updated successfully, but these errors were encountered: