Skip to content

Commit

Permalink
Exception handling O(n) -> O(1) x 1.
Browse files Browse the repository at this point in the history 
This commit is the first in a commit chain fundamentally refactoring
the private `beartype._decor._code._pep._error` subpackage responsible
for raising human-readable exceptions in the event of type-checking
violations from a linear- to constant-time algorithm, while still
preserving optional support for linear-time behaviour in anticipation of
permitting end users to conditionally enable linear-time type-checking
under some subsequent stable release of `beartype`. Specifically, this
commit refactors our O(1) type-checker to pass all metadata required by
our exception handler to conditionally switch between O(1) and O(n)
behaviour as required. Naturally, our exception handler has yet to
actually use this metadata in any meaningful way. (*Automatic dogmas!*)
  • Loading branch information
@leycec
leycec committed Feb 2, 2021
1 parent 5e01040 commit e65626c
Show file tree
Hide file tree
Showing 79 changed files with 238 additions and 133 deletions.
26 changes: 14 additions & 12 deletions README.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1469,7 +1469,7 @@ When should I use beartype?
Use ``beartype`` to assure the quality of Python code beyond what tests alone
can assure. If you have yet to test, do that first with a pytest_-based test
suite, tox_ configuration, and `continuous integration (CI) <continuous
integration_>`__ configuration. If you have any time or money left, `annotate
integration_>`__. If you have any time, money, or motivation left, `annotate
callables with PEP-compliant type hints <Compliance_>`__ and `decorate those
callables with the @beartype.beartype decorator <Usage_>`__.

Expand All @@ -1478,23 +1478,23 @@ over the objects passed to or returned from your callables – *especially*
whenever you cannot limit the size of those objects. This includes common
developer scenarios like:

* You are the author of an open-source library intended to be reused by a
* You are the author of an **open-source library** intended to be reused by a
general audience.
* You are the author of an open- or closed-source app either accepting as input
or generating as output arbitrary (or at least sufficiently large) data, some
of which invariably filters down into callable parameter and return values.
* You are the author of a **public app** accepting as input or generating as
output sufficiently large data internally passed to or returned from app
callables.

Prefer ``beartype`` over static type checkers whenever:

* You want to type-check `types decidable only at runtime <Versus Static Type
* You want to `check types decidable only at runtime <Versus Static Type
Checkers_>`__.
* You want to JIT_ your Python under PyPy_ (which you should), which most
static type checkers are currently incompatible with.
* You want to JIT_ your code with PyPy_, :superscript:`...which you should`,
which most static type checkers remain incompatible with.

Even where none of the prior apply, still consider ``beartype``. It's
`cost-free at both installation- and runtime <Overview_>`__. Leverage
``beartype`` until you find something that suites you better, because
``beartype`` is *always* better than nothing.
Even where none of the above apply, still use ``beartype``. It's `free as in
beer and speech <gratis versus libre_>`__ and `cost-free at installation- and
runtime <Overview_>`__. Leverage ``beartype`` until you find something that
suites you better, because ``beartype`` is *always* better than nothing.

Why should I use beartype?
--------------------------
Expand Down Expand Up @@ -2875,6 +2875,8 @@ application stack at tool rather than Python runtime) include:
https://en.wikipedia.org/wiki/Continuous_integration
.. _duck typing:
https://en.wikipedia.org/wiki/Duck_typing
.. _gratis versus libre:
https://en.wikipedia.org/wiki/Gratis_versus_libre
.. _memory safety:
https://en.wikipedia.org/wiki/Memory_safety
.. _random walk:
Expand Down
38 changes: 38 additions & 0 deletions beartype/_decor/_code/_pep/_error/peperror.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,18 @@
This private submodule is *not* intended for importation by downstream callers.
'''

#FIXME: *WOOPS.* We tragically realize we need to significantly refactor this
#entire package for O(1) behaviour, because... I mean, really. Consider large
#nested lists, yes? That's all we need to say. Specifically:
#
#* Refactor all private submodules of this package as follows:
# * If the "random_int" parameter is non-"None", specifically type-check
# *ONLY* the sole index of the current container indicated by that
# parameter.
# * Else, continue to perform O(n)-style type-checking as we currently do.

# ....................{ IMPORTS }....................
from beartype.cave import NoneTypeOr
from beartype.meta import URL_ISSUES
from beartype.roar import (
BeartypeCallHintPepParamException,
Expand Down Expand Up @@ -79,9 +90,13 @@

# ....................{ RAISERS }....................
def raise_pep_call_exception(
# Mandatory parameters.
func: 'CallableTypes',
pith_name: str,
pith_value: object,

# Optional parameters.
random_int: 'Optional[int]' = None,
) -> None:
'''
Raise a human-readable exception detailing the failure of the parameter
Expand Down Expand Up @@ -129,6 +144,27 @@ def raise_pep_call_exception(
value returned from this callable.
pith_value : object
Passed parameter or returned value failing to satisfy this hint.
random_int: Optional[int]
**Pseudo-random integer** (i.e., unsigned 32-bit integer
pseudo-randomly generated by the parent :func:`beartype.beartype`
wrapper function in type-checking randomly indexed container items by
the current call to that function) if that function generated such an
integer *or* ``None`` otherwise (i.e., if that function generated *no*
such integer). Note that this parameter critically governs whether this
exception handler runs in constant or linear time. Specifically, if
this parameter is:
* An integer, this handler runs in **constant time.** Since there
exists a one-to-one relation between this integer and the random
container item(s) type-checked by the parent
:func:`beartype.beartype` wrapper function, receiving this integer
enables this handler to efficiently re-type-check the same random
container item(s) type-checked by the parent in constant time rather
type-checking all container items in linear time.
* ``None``, this handler runs in **linear time.**
Defaults to ``None``, implying this exception handler runs in linear
time by default.
Raises
----------
Expand All @@ -152,6 +188,8 @@ def raise_pep_call_exception(
'''
assert callable(func), f'{repr(func)} uncallable.'
assert isinstance(pith_name, str), f'{repr(pith_name)} not string.'
assert isinstance(random_int, NoneTypeOr[int]), (
f'{repr(random_int)} not integer or "None".')
# print('''raise_pep_call_exception(
# func={!r},
# pith_name={!r},
Expand Down
31 changes: 23 additions & 8 deletions beartype/_decor/_code/_pep/_pephint.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -770,7 +770,9 @@
)
from beartype._decor._code.codesnip import CODE_INDENT_1, CODE_INDENT_2
from beartype._decor._code._pep._pepsnip import (
PEP_CODE_CHECK_HINT_ROOT,
PEP_CODE_CHECK_HINT_GENERIC_PREFIX,
PEP_CODE_CHECK_HINT_GENERIC_SUFFIX,
PEP_CODE_CHECK_HINT_ROOT_PREFIX,
PEP_CODE_CHECK_HINT_TUPLE_FIXED_PREFIX,
PEP_CODE_CHECK_HINT_TUPLE_FIXED_SUFFIX,
PEP_CODE_HINT_CHILD_PLACEHOLDER_PREFIX,
Expand All @@ -779,21 +781,21 @@
PEP_CODE_HINT_FORWARDREF_UNQUALIFIED_PLACEHOLDER_SUFFIX,
PEP_CODE_PITH_NAME_PREFIX,
PEP_CODE_PITH_ROOT_NAME,
PEP_CODE_CHECK_HINT_GENERIC_PREFIX,
PEP_CODE_CHECK_HINT_GENERIC_SUFFIX,
PEP_CODE_RAISE_PEP_CALL_EXCEPTION_RANDOM_INT,
PEP484_CODE_CHECK_HINT_UNION_PREFIX,
PEP484_CODE_CHECK_HINT_UNION_SUFFIX,

# Bound format methods.
PEP_CODE_CHECK_HINT_NONPEP_TYPE_format,
PEP_CODE_CHECK_HINT_ROOT_SUFFIX_format,
PEP_CODE_CHECK_HINT_SEQUENCE_STANDARD_format,
PEP_CODE_CHECK_HINT_SEQUENCE_STANDARD_PITH_CHILD_EXPR_format,
PEP_CODE_CHECK_HINT_TUPLE_FIXED_EMPTY_format,
PEP_CODE_CHECK_HINT_TUPLE_FIXED_LEN_format,
PEP_CODE_CHECK_HINT_TUPLE_FIXED_NONEMPTY_CHILD_format,
PEP_CODE_CHECK_HINT_TUPLE_FIXED_NONEMPTY_PITH_CHILD_EXPR_format,
PEP_CODE_PITH_ASSIGN_EXPR_format,
PEP_CODE_CHECK_HINT_GENERIC_CHILD_format,
PEP_CODE_PITH_ASSIGN_EXPR_format,
PEP484_CODE_CHECK_HINT_UNION_CHILD_PEP_format,
PEP484_CODE_CHECK_HINT_UNION_CHILD_NONPEP_format,
)
Expand Down Expand Up @@ -1404,8 +1406,8 @@ def _enqueue_hint_child(pith_child_expr: str) -> str:
# Python code snippet type-checking the root pith against the root hint,
# localized separately from the "func_code" snippet to enable this function
# to validate this code to be valid *BEFORE* returning this code.
func_root_code = PEP_CODE_CHECK_HINT_ROOT.format(
hint_child_placeholder=hint_child_placeholder)
func_root_code = (
f'{PEP_CODE_CHECK_HINT_ROOT_PREFIX}{hint_child_placeholder}')

# Python code snippet to be returned, seeded with a placeholder to be
# subsequently replaced on the first iteration of the breadth-first search
Expand Down Expand Up @@ -1604,7 +1606,7 @@ def _enqueue_hint_child(pith_child_expr: str) -> str:
# the source code for non-file-based modules) and possibly even
# go so far as to define a PEP 302-compatible beartype module
# loader. Clearly, that's out of scope. For now, this suffices.
#* In the "beartype_test.unit.data._data_hint_pep" submodule:
#* In the "beartype_test.a00_unit.data._data_hint_pep" submodule:
# * Add a new "_PepHintMetadata.code_str_match_regexes" field,
# defined as an iterable of regular expressions matching
# substrings of the "func_wrapper.__beartype_wrapper_code"
Expand All @@ -1614,7 +1616,7 @@ def _enqueue_hint_child(pith_child_expr: str) -> str:
# * For deeply nested "HINTS_PEP_META" entries, define this
# field as follows:
# code_str_match_regexes=(r'\s+:=\s+',)
#* In the "beartype_test.unit.pep.p484.test_p484" submodule:
#* In the "beartype_test.a00_unit.pep.p484.test_p484" submodule:
# * Match the "pep_hinted.__beartype_wrapper_code" string against
# all regular expressions in the "code_str_match_regexes"
# iterable for the currently iterated "pep_hint_meta".
Expand Down Expand Up @@ -2554,6 +2556,19 @@ def _enqueue_hint_child(pith_child_expr: str) -> str:
f'{hint_root_label} {repr(hint_root)} not type-checked.')
# Else, the breadth-first search above successfully generated code.

# Suffix this code by a Python code snippet raising a human-readable
# exception when the root pith violates the root type hint.
func_code += PEP_CODE_CHECK_HINT_ROOT_SUFFIX_format(
random_int_if_any=(
# If type-checking the root pith requires a pseudo-random integer,
# pass this integer to the function raising this exception.
PEP_CODE_RAISE_PEP_CALL_EXCEPTION_RANDOM_INT
if is_func_code_needs_random_int else
# Else, call that function *WITHOUT* passing that integer.
''
),
)

# Tuple of the unqualified classnames referred to by all relative forward
# references visitable from this root hint converted from this set to
# reduce space consumption after memoization by @callable_cached.
Expand Down
74 changes: 56 additions & 18 deletions beartype/_decor/_code/_pep/_pepsnip.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,11 @@

# ....................{ IMPORTS }....................
from beartype._decor._code.codesnip import (
PARAM_NAME_FUNC, PARAM_NAME_TYPISTRY)
ARG_NAME_FUNC,
ARG_NAME_TYPISTRY,
VAR_NAME_ARGS_LEN,
VAR_NAME_RANDOM_INT,
)
from inspect import Parameter

# ....................{ PITH }....................
Expand Down Expand Up @@ -87,22 +91,22 @@
# Localize this positional or keyword parameter if passed *OR* to the
# sentinel value "__beartypistry" guaranteed to never be passed otherwise.
{PEP_CODE_PITH_ROOT_NAME} = (
args[{{arg_index}}] if __beartype_args_len > {{arg_index}} else
kwargs.get({{arg_name!r}}, {PARAM_NAME_TYPISTRY})
args[{{arg_index}}] if {VAR_NAME_ARGS_LEN} > {{arg_index}} else
kwargs.get({{arg_name!r}}, {ARG_NAME_TYPISTRY})
)
# If this parameter was passed...
if {PEP_CODE_PITH_ROOT_NAME} is not {PARAM_NAME_TYPISTRY}:''',
if {PEP_CODE_PITH_ROOT_NAME} is not {ARG_NAME_TYPISTRY}:''',

# Snippet localizing any keyword-only parameter (e.g., "*, kwarg") by
# lookup in the wrapper's variadic "**kwargs" dictionary. (See above.)
Parameter.KEYWORD_ONLY: f'''
# Localize this keyword-only parameter if passed *OR* to the sentinel value
# "__beartypistry" guaranteed to never be passed otherwise.
{PEP_CODE_PITH_ROOT_NAME} = kwargs.get({{arg_name!r}}, {PARAM_NAME_TYPISTRY})
{PEP_CODE_PITH_ROOT_NAME} = kwargs.get({{arg_name!r}}, {ARG_NAME_TYPISTRY})
# If this parameter was passed...
if {PEP_CODE_PITH_ROOT_NAME} is not {PARAM_NAME_TYPISTRY}:''',
if {PEP_CODE_PITH_ROOT_NAME} is not {ARG_NAME_TYPISTRY}:''',

# Snippet iteratively localizing all variadic positional parameters.
Parameter.VAR_POSITIONAL: f'''
Expand All @@ -119,7 +123,7 @@
PEP_CODE_CHECK_RETURN_PREFIX = f'''
# Call this function with all passed parameters and localize the value
# returned from this call.
{PEP_CODE_PITH_ROOT_NAME} = {PARAM_NAME_FUNC}(*args, **kwargs)
{PEP_CODE_PITH_ROOT_NAME} = {ARG_NAME_FUNC}(*args, **kwargs)
# Noop required to artifically increase indentation level. Note that
# CPython implicitly optimizes this conditional away - which is nice.
Expand Down Expand Up @@ -159,13 +163,13 @@
PEP484_CODE_CHECK_NORETURN = f'''
# Call this function with all passed parameters and localize the value
# returned from this call.
{PEP_CODE_PITH_ROOT_NAME} = {PARAM_NAME_FUNC}(*args, **kwargs)
{PEP_CODE_PITH_ROOT_NAME} = {ARG_NAME_FUNC}(*args, **kwargs)
# Since this function annotated by "typing.NoReturn" successfully returned
# a value rather than raising an exception or halting the active Python
# interpreter, unconditionally raise an exception.
__beartype_raise_pep_call_exception(
func={PARAM_NAME_FUNC},
func={ARG_NAME_FUNC},
pith_name={PEP_CODE_PITH_ROOT_PARAM_NAME_PLACEHOLDER},
pith_value={PEP_CODE_PITH_ROOT_NAME},
)'''
Expand Down Expand Up @@ -218,27 +222,59 @@
'''

# ....................{ HINT ~ pith : root }....................
PEP_CODE_CHECK_HINT_ROOT = f'''
PEP_CODE_CHECK_HINT_ROOT_PREFIX = f'''
# Type-check this passed parameter or return value against this
# PEP-compliant type hint.
if not {{hint_child_placeholder}}:
if not '''
'''
PEP-compliant code snippet prefixing all code type-checking the **root pith**
(i.e., value of the current parameter or return value) against the root
PEP-compliant type hint annotating that pith.
This prefix is intended to be locally suffixed in the
:func:`beartype._decor._code._pep._pephint.pep_code_check_hint` function by:
#. The value of the ``hint_child_placeholder`` local variable.
#. The :data:`PEP_CODE_CHECK_HINT_ROOT_SUFFIX` suffix.
'''


PEP_CODE_CHECK_HINT_ROOT_SUFFIX = f''':
__beartype_raise_pep_call_exception(
func={PARAM_NAME_FUNC},
func={ARG_NAME_FUNC},
pith_name={PEP_CODE_PITH_ROOT_PARAM_NAME_PLACEHOLDER},
pith_value={PEP_CODE_PITH_ROOT_NAME},
pith_value={PEP_CODE_PITH_ROOT_NAME},{{random_int_if_any}}
)
'''
'''
PEP-compliant code snippet type-checking the **root pith** (i.e., value of the
current parameter or return value) against the root PEP-compliant type hint
annotating that pith.
PEP-compliant code snippet suffixing all code type-checking the **root pith**
(i.e., value of the current parameter or return value) against the root
PEP-compliant type hint annotating that pith.
This snippet expects to be formatted with these named interpolations:
* ``{random_int_if_any}``, whose value is either:
* If type-checking the current type hint requires a pseudo-random integer,
:data:`PEP_CODE_RAISE_PEP_CALL_EXCEPTION_RANDOM_INT`.
* Else, the empty substring.
Design
----------
**This string is the only code snippet defined by this submodule to raise an
exception.** All other such snippets only test the current pith against the
current child PEP-compliant type hint and are thus intended to be dynamically
embedded
embedded in the conditional test initiated by the
:data:`PEP_CODE_CHECK_HINT_ROOT_PREFIX` code snippet.
'''


PEP_CODE_RAISE_PEP_CALL_EXCEPTION_RANDOM_INT = f'''
random_int={VAR_NAME_RANDOM_INT},'''
'''
PEP-compliant code snippet suffixing all code type-checking the **root pith**
(i.e., value of the current parameter or return value) against the root
PEP-compliant type hint annotating that pith.
'''

# ....................{ HINT ~ nonpep }....................
Expand Down Expand Up @@ -351,7 +387,7 @@


PEP_CODE_CHECK_HINT_SEQUENCE_STANDARD_PITH_CHILD_EXPR = (
'''{pith_curr_assigned_expr}[__beartype_random_int % len({pith_curr_assigned_expr})]''')
f'''{{pith_curr_assigned_expr}}[{VAR_NAME_RANDOM_INT} % len({{pith_curr_assigned_expr}})]''')
'''
PEP-compliant Python expression yielding the value of a randomly indexed item
of the current pith (which, by definition, *must* be a standard sequence).
Expand Down Expand Up @@ -495,6 +531,8 @@
PEP_CODE_CHECK_HINT_NONPEP_TYPE.format)
PEP_CODE_CHECK_HINT_GENERIC_CHILD_format = (
PEP_CODE_CHECK_HINT_GENERIC_CHILD.format)
PEP_CODE_CHECK_HINT_ROOT_SUFFIX_format = (
PEP_CODE_CHECK_HINT_ROOT_SUFFIX.format)
PEP_CODE_CHECK_HINT_SEQUENCE_STANDARD_format = (
PEP_CODE_CHECK_HINT_SEQUENCE_STANDARD.format)
PEP_CODE_CHECK_HINT_SEQUENCE_STANDARD_PITH_CHILD_EXPR_format = (
Expand Down
4 changes: 2 additions & 2 deletions beartype/_decor/_code/codemain.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -567,7 +567,7 @@
# ....................{ IMPORTS }....................
from beartype.roar import BeartypeDecorParamNameException
from beartype._decor._code.codesnip import (
CODE_INIT_PARAMS_POSITIONAL_LEN,
CODE_INIT_ARGS_LEN,
CODE_INIT_RANDOM_INT,
CODE_RETURN_UNCHECKED,
CODE_SIGNATURE,
Expand Down Expand Up @@ -905,7 +905,7 @@ def _code_check_params(data: BeartypeData) -> 'Tuple[str, bool]':
# If this callable accepts one or more positional parameters, this
# snippet preceded by code localizing the number of these
# parameters.
f'{CODE_INIT_PARAMS_POSITIONAL_LEN}{func_code}'
f'{CODE_INIT_ARGS_LEN}{func_code}'
if is_params_positional else
# Else, this callable accepts *NO* positional parameters. In this
# case, this snippet as is.
Expand Down

0 comments on commit e65626c

Please sign in to comment.