Skip to content

Commit

Permalink
Ignorable PEP 695 type aliases.
Browse files Browse the repository at this point in the history
This commit adds support for **ignorable PEP 695-compliant `type`
aliases** (e.g., `type this_is_horrible = object`,
`type this_is_even_worse = int | str | Any`), resolving a future issue
that somebody was surely about to submit. Relatedly, this commit also
significantly optimizes @beartype's internal detection of ignorable type
hints. Previously, that detection inefficiently operated in `O(n)` time
for `n` the number of PEP-specific detection routines; since the number
of PEPs supported by @beartype has blossomed out of control (...*a good
thing, mostly*), so too has the number of PEP-specific detection
routines and thus @beartype's internal detection of ignorable type
hints. Now, that detection efficiently operates in `O(1)` time. Since
@beartype repeatedly performed that detection during dynamic code
generation, this constitutes a significant decorator-time optimization.
(*Formidably hearty formulations of Romulan LAN parties!*)
  • Loading branch information
leycec committed Feb 22, 2024
1 parent 22787fb commit 16a835c
Show file tree
Hide file tree
Showing 17 changed files with 596 additions and 437 deletions.
63 changes: 37 additions & 26 deletions beartype/_check/convert/convreduce.py
Original file line number Diff line number Diff line change
Expand Up @@ -219,9 +219,9 @@ def _reduce_hint_uncached(

# If a callable reducing hints of this sign was previously registered,
# reduce this hint to another hint via this callable.
if hint_reducer is not None:
if hint_reducer is not None: # type: ignore[call-arg]
# print(f'Reducing hint {repr(hint)} to...')
hint = hint_reducer( # type: ignore[call-arg]
hint = hint_reducer(
hint=hint, # pyright: ignore[reportGeneralTypeIssues]
conf=conf,
cls_stack=cls_stack,
Expand Down Expand Up @@ -308,22 +308,35 @@ def _reduce_hint_cached(
return hint

# ....................{ PRIVATE ~ hints }....................
_DictReducer = Dict[Optional[HintSign], Callable]
# Note that these type hints would ideally be defined with the mypy-specific
# "callback protocol" pseudostandard, documented here:
# https://mypy.readthedocs.io/en/stable/protocols.html#callback-protocols
#
# Doing so would enable static type-checkers to type-check that the values of
# these dictionaries are valid reducer functions. Sadly, that pseudostandard is
# absurdly strict to the point of practical uselessness. Attempting to conform
# to that pseudostandard would require refactoring *ALL* reducer functions to
# explicitly define the same signature. However, we have intentionally *NOT*
# done that. Why? Doing so would substantially increase the fragility of this
# API by preventing us from readily adding and removing infrequently required
# parameters (e.g., "cls_stack", "pith_name"). Callback protocols suck, frankly.
_HintSignToReduceHintCached = Dict[Optional[HintSign], Callable]
'''
PEP-compliant type hint matching a **cached reducer dictionary** (i.e.,
mapping from each sign uniquely identifying various type hints to a memoized
callable reducing those higher- to lower-level hints).
'''


_HintSignToReduceHintUncached = _HintSignToReduceHintCached
'''
PEP-compliant type hint matching a **reducer dictionary** (i.e., dictionary
mapping from each sign uniquely identifying various type hints to a callable
reducing those higher- to lower-level hints).
PEP-compliant type hint matching an **uncached reducer dictionary** (i.e.,
mapping from each sign uniquely identifying various type hints to an unmemoized
callable reducing those higher- to lower-level hints).
'''

# ....................{ PRIVATE ~ dicts }....................
#FIXME: After dropping Python 3.7 support:
#* Replace "Callable[[object, str], object]" here with a callback protocol:
# https://mypy.readthedocs.io/en/stable/protocols.html#callback-protocols
# Why? Because the current approach forces positional arguments. But we call
# these callables with keyword arguments above! Chaos ensues.
#* Remove the "# type: ignore[call-arg]" pragmas above, which are horrible.
#* Remove the "# type: ignore[dict-item]" pragmas above, which are horrible.
_HINT_SIGN_TO_REDUCE_HINT_CACHED: _DictReducer = {
_HINT_SIGN_TO_REDUCE_HINT_CACHED: _HintSignToReduceHintCached = {
# ..................{ NON-PEP }..................
# If this hint is identified by *NO* sign, this hint is either an
# isinstanceable type *OR* a hint unrecognized by beartype. In either case,
Expand Down Expand Up @@ -466,8 +479,7 @@ def _reduce_hint_cached(
by the :func:`.callable_cached` decorator reducing those higher- to lower-level
hints).
Each value of this dictionary should be a valid reducer, defined as a function
with signature resembling:
Each value of this dictionary is expected to have a signature resembling:
.. code-block:: python
Expand All @@ -488,19 +500,18 @@ def reduce_hint_pep{pep_number}(
expected sign. By design, a reducer is only ever passed a type hint of the
expected sign.
* Reducers should *not* be memoized (e.g., by the
:func:`.callable_cached` decorator). Since the higher-level
:func:`.reduce_hint` function that is the sole entry point to calling all
lower-level reducers is itself memoized, reducers themselves do neither
require nor benefit from memoization. Moreover, even if they did either
require or benefit from memoization, they couldn't be -- at least, not
directly. Why? Because :func:`.reduce_hint` necessarily passes keyword
arguments to all reducers. But memoized functions *cannot* receive keyword
arguments (without destroying efficiency and thus the entire impetus for
memoization). Ergo, reducers *cannot* be memoized.
``callable_cached`` decorator). Since the higher-level :func:`.reduce_hint`
function that is the sole entry point to calling all lower-level reducers is
itself memoized, reducers themselves neither require nor benefit from
memoization. Moreover, even if they did either require or benefit from
memoization, they couldn't be -- at least, not directly. Why? Because
:func:`.reduce_hint` necessarily passes keyword arguments to all reducers. But
memoized functions *cannot* receive keyword arguments (without destroying
efficiency and thus the entire point of memoization).
'''


_HINT_SIGN_TO_REDUCE_HINT_UNCACHED: _DictReducer = {
_HINT_SIGN_TO_REDUCE_HINT_UNCACHED: _HintSignToReduceHintUncached = {
# ..................{ PEP 647 }..................
# If this hint is a PEP 647-compliant "typing.TypeGuard[...]" type hint,
# either:
Expand Down
260 changes: 122 additions & 138 deletions beartype/_util/hint/pep/proposal/pep484/utilpep484.py
Original file line number Diff line number Diff line change
Expand Up @@ -11,19 +11,11 @@

# ....................{ IMPORTS }....................
from beartype._cave._cavefast import NoneType
from beartype._data.hint.pep.sign.datapepsigncls import HintSign
from beartype._data.hint.pep.sign.datapepsigns import (
HintSignGeneric,
HintSignNewType,
HintSignTypeVar,
)
from beartype._data.hint.pep.sign.datapepsignset import HINT_SIGNS_UNION

# Intentionally import PEP 484-compliant "typing" type hint factories rather
# than possibly PEP 585-compliant "beartype.typing" type hint factories.
from typing import (
Generic,
Optional,
Tuple,
)

Expand All @@ -34,154 +26,146 @@
'''

# ....................{ TESTERS ~ ignorable }....................
def is_hint_pep484_ignorable_or_none(
hint: object, hint_sign: HintSign) -> Optional[bool]:
#FIXME: Shift into a more appropriate submodule, please.
def is_hint_pep484585_generic_ignorable(hint: object) -> bool:
'''
``True`` only if the passed object is a :pep:`484`-compliant **ignorable
type hint,** ``False`` only if this object is a :pep:`484`-compliant
unignorable type hint, and ``None`` if this object is *not* `PEP
484`_-compliant.
Specifically, this tester function returns ``True`` only if this object is
a deeply ignorable :pep:`484`-compliant type hint, including:
* A parametrization of the :class:`typing.Generic` abstract base class (ABC)
by one or more type variables. As the name implies, this ABC is generic
and thus fails to impose any meaningful constraints. Since a type variable
in and of itself also fails to impose any meaningful constraints, these
parametrizations are safely ignorable in all possible contexts: e.g.,
.. code-block:: python
from typing import Generic, TypeVar
T = TypeVar('T')
def noop(param_hint_ignorable: Generic[T]) -> T: pass
* The :func:`NewType` closure factory function passed an ignorable child
type hint. Unlike most :mod:`typing` constructs, that function does *not*
cache the objects it returns: e.g.,
.. code-block:: python
>>> from typing import NewType
>>> NewType('TotallyNotAStr', str) is NewType('TotallyNotAStr', str)
False
Since this implies every call to ``NewType({same_name}, object)`` returns
a new closure, the *only* means of ignoring ignorable new type aliases is
dynamically within this function.
* The :data:`Optional` or :data:`Union` singleton subscripted by one or
more ignorable type hints (e.g., ``typing.Union[typing.Any, bool]``).
Why? Because unions are by definition only as narrow as their widest
child hint. However, shallowly ignorable type hints are ignorable
precisely because they are the widest possible hints (e.g.,
:class:`object`, :attr:`typing.Any`), which are so wide as to constrain
nothing and convey no meaningful semantics. A union of one or more
shallowly ignorable child hints is thus the widest possible union,
which is so wide as to constrain nothing and convey no meaningful
semantics. Since there exist a countably infinite number of possible
:data:`Union` subscriptions by one or more ignorable type hints, these
subscriptions *cannot* be explicitly listed in the
:data:`HINTS_REPR_IGNORABLE_SHALLOW` frozenset. Instead, these
subscriptions are dynamically detected by this tester at runtime and thus
referred to as **deeply ignorable type hints.**
:data:`True` only if the passed :pep:`484`- or :pep:`585`-compliant generic
is ignorable.
Specifically, this tester ignores *all* parametrizations of the
:class:`typing.Generic` abstract base class (ABC) by one or more type
variables. As the name implies, this ABC is generic and thus fails to impose
any meaningful constraints. Since a type variable in and of itself also
fails to impose any meaningful constraints, these parametrizations are
safely ignorable in all possible contexts: e.g.,
.. code-block:: python
from typing import Generic, TypeVar
T = TypeVar('T')
def noop(param_hint_ignorable: Generic[T]) -> T: pass
This tester is intentionally *not* memoized (e.g., by the
:func:`callable_cached` decorator), as this tester is only safely callable
by the memoized parent
``callable_cached`` decorator), as this tester is only safely callable by
the memoized parent
:func:`beartype._util.hint.utilhinttest.is_hint_ignorable` tester.
Parameters
----------
hint : object
Type hint to be inspected.
hint_sign : HintSign
**Sign** (i.e., arbitrary object uniquely identifying this hint).
Returns
-------
Optional[bool]
Either:
* If this object is :pep:`484`-compliant:
* If this object is a ignorable, ``True``.
* Else, ``False``.
* If this object is *not* :pep:`484`-compliant, ``None``.
bool
:data:`True` only if this :pep:`484`-compliant type hint is ignorable.
'''
# print(f'Testing PEP 484 hint {repr(hint)} [{repr(hint_sign)}] deep ignorability...')

#FIXME: Remove this *AFTER* properly supporting type variables. For
#now, ignoring type variables is required ta at least shallowly support
#generics parametrized by one or more type variables.
# Avoid circular import dependencies.
from beartype._util.hint.pep.utilpepget import get_hint_pep_origin_or_none
# print(f'Testing generic hint {repr(hint)} deep ignorability...')

# For minor efficiency gains, the following tests are intentionally ordered
# in descending likelihood of a match.
# If this generic is the "typing.Generic" superclass directly parametrized
# by one or more type variables (e.g., "typing.Generic[T]"), return true.
#
# If this hint is a PEP 484-compliant type variable, unconditionally return
# true. Type variables require non-trivial and currently unimplemented
# decorator support.
if hint_sign is HintSignTypeVar:
# Note that we intentionally avoid calling the
# get_hint_pep_origin_type_isinstanceable_or_none() function here, which has
# been intentionally designed to exclude PEP-compliant type hints
# originating from "typing" type origins for stability reasons.
if get_hint_pep_origin_or_none(hint) is Generic:
# print(f'Testing generic hint {repr(hint)} deep ignorability... True')
return True
# Else, this hint is *NOT* a PEP 484-compliant type variable.
#
# If this hint is a PEP 484-compliant union...
elif hint_sign in HINT_SIGNS_UNION:
# Avoid circular import dependencies.
from beartype._util.hint.pep.utilpepget import get_hint_pep_args
from beartype._util.hint.utilhinttest import is_hint_ignorable

# Return true only if one or more child hints of this union are
# recursively ignorable. See the function docstring.
return any(
is_hint_ignorable(hint_child)
for hint_child in get_hint_pep_args(hint)
)
# Else, this hint is *NOT* a PEP 484-compliant union.
#
# If this hint is a PEP 484-compliant generic...
elif hint_sign is HintSignGeneric:
# Avoid circular import dependencies.
from beartype._util.hint.pep.utilpepget import (
get_hint_pep_origin_or_none)

# print(f'Testing generic hint {repr(hint)} deep ignorability...')
# If this generic is the "typing.Generic" superclass directly
# parametrized by one or more type variables (e.g.,
# "typing.Generic[T]"), return true.
#
# Note that we intentionally avoid calling the
# get_hint_pep_origin_type_isinstanceable_or_none() function here, which
# has been intentionally designed to exclude PEP-compliant type hints
# originating from "typing" type origins for stability reasons.
if get_hint_pep_origin_or_none(hint) is Generic:
# print(f'Testing generic hint {repr(hint)} deep ignorability... True')
return True
# Else, this generic is *NOT* the "typing.Generic" superclass directly
# parametrized by one or more type variables and thus *NOT* an
# ignorable non-protocol.
#
# Note that this condition being false is *NOT* sufficient to declare
# this hint to be unignorable. Notably, the type origin originating
# both ignorable and unignorable protocols is "Protocol" rather than
# "Generic". Ergo, this generic could still be an ignorable protocol.
# print(f'Testing generic hint {repr(hint)} deep ignorability... False')
# Else, this hint is *NOT* a PEP 484-compliant generic.
# Else, this generic is *NOT* the "typing.Generic" superclass directly
# parametrized by one or more type variables and thus *NOT* an ignorable
# non-protocol.
#
# If this hint is a PEP 484-compliant new type...
elif hint_sign is HintSignNewType:
# Avoid circular import dependencies.
from beartype._util.hint.utilhinttest import is_hint_ignorable
from beartype._util.hint.pep.proposal.pep484.utilpep484newtype import (
get_hint_pep484_newtype_alias)

# Return true only if this hint aliases an ignorable child type hint.
return is_hint_ignorable(get_hint_pep484_newtype_alias(hint))
# Else, this hint is *NOT* a PEP 484-compliant new type.

# Return "None", as this hint is unignorable only under PEP 484.
return None
# Note that this condition being false is *NOT* sufficient to declare this
# hint to be unignorable. Notably, the origin type originating both
# ignorable and unignorable protocols is "Protocol" rather than "Generic".
# Ergo, this generic could still be an ignorable protocol.
# print(f'Testing generic hint {repr(hint)} deep ignorability... False')

#FIXME: Probably insufficient. *shrug*
return False


#FIXME: Remove this *AFTER* properly supporting type variables. For now,
#ignoring type variables is required ta at least shallowly support generics
#parametrized by one or more type variables.
def is_hint_pep484_typevar_ignorable(hint: object) -> bool:
'''
:data:`True` unconditionally.
This tester currently unconditionally ignores *all* :pep:`484`-compliant
type variables, which require non-trivial and currently unimplemented
code generation support.
This tester is intentionally *not* memoized (e.g., by the
``callable_cached`` decorator), as this tester is only safely callable by
the memoized parent
:func:`beartype._util.hint.utilhinttest.is_hint_ignorable` tester.
Parameters
----------
hint : object
Type hint to be inspected.
Returns
-------
bool
:data:`True` only if this :pep:`484`-compliant type hint is ignorable.
'''

# Ignore *ALL* PEP 484-compliant type variables.
return True


#FIXME: Shift into a more appropriate submodule, please.
def is_hint_pep484604_union_ignorable(hint: object) -> bool:
'''
:data:`True` only if the passed :pep:`484`- or :pep:`604`-compliant union is
ignorable.
Specifically, this tester ignores the :obj:`typing.Optional` or
:obj:`typing.Union` singleton subscripted by one or more ignorable type
hints (e.g., ``typing.Union[typing.Any, bool]``). Why? Because unions are by
definition only as narrow as their widest child hint. However, shallowly
ignorable type hints are ignorable precisely because they are the widest
possible hints (e.g., :class:`object`, :attr:`typing.Any`), which are so
wide as to constrain nothing and convey no meaningful semantics. A union of
one or more shallowly ignorable child hints is thus the widest possible
union, which is so wide as to constrain nothing and convey no meaningful
semantics. Since there exist a countably infinite number of possible
:data:`Union` subscriptions by one or more ignorable type hints, these
subscriptions *cannot* be explicitly listed in the
:data:`beartype._data.hint.pep.datapeprepr.HINTS_REPR_IGNORABLE_SHALLOW`
frozenset. Instead, these subscriptions are dynamically detected by this
tester at runtime and thus referred to as **deeply ignorable type hints.**
This tester is intentionally *not* memoized (e.g., by the
``callable_cached`` decorator), as this tester is only safely callable by
the memoized parent
:func:`beartype._util.hint.utilhinttest.is_hint_ignorable` tester.
Parameters
----------
hint : object
Type hint to be inspected.
Returns
-------
bool
:data:`True` only if this :pep:`484`-compliant type hint is ignorable.
'''

# Avoid circular import dependencies.
from beartype._util.hint.pep.utilpepget import get_hint_pep_args
from beartype._util.hint.utilhinttest import is_hint_ignorable

# Return true only if one or more child hints of this union are recursively
# ignorable. See the function docstring.
return any(
is_hint_ignorable(hint_child) for hint_child in get_hint_pep_args(hint))

# ....................{ REDUCERS }....................
#FIXME: Replace the ambiguous parameter:
Expand Down

0 comments on commit 16a835c

Please sign in to comment.