Skip to content

Commit

Permalink
PEP 561 x 5.
Browse files Browse the repository at this point in the history
This commit is the next in a commit chain rendering `beartype` compliant
with PEP 561 by annotating the codebase in a manner specifically
compatible with the most popular third-party static type checker, mypy,
en-route to eventually resolving issue #25 kindly submitted also by best
macOS package manager ever @harens. Specifically, this commit resolves
three pending micro-issues astutely uncovered by @harens:

* The failure to prefix an include path by `include ` in `MANIFEST.in`.
* The failure to type-hint the top-level `@beartype.beartype decorator.
* The failure to export that decorator from `beartype.__init__.__all__`.

(*Effervescent arborescence!*)
  • Loading branch information
leycec committed Feb 19, 2021
1 parent ceee48b commit 2c9c92f
Show file tree
Hide file tree
Showing 49 changed files with 229 additions and 209 deletions.
2 changes: 1 addition & 1 deletion MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ include setup.py
include tox.ini

# Include all requisite package-level install-time files.
beartype/py.typed
include beartype/py.typed

# ....................{ INCLUDE ~ recursive }....................
# Include all requisite project-specific test packages.
Expand Down
2 changes: 0 additions & 2 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -342,8 +342,6 @@ whenever:

* You want to `check types decidable only at runtime <Versus Static Type
Checkers_>`__.
* You want to JIT_ your code with PyPy_, :superscript:`...which you should`,
which most static type checkers remain incompatible with.
* You prefer to write code rather than fight a static type checker, because
static type inference of a dynamically-typed language is guaranteed to fail
(and frequently does). If you've ever cursed the sky after suffixing working
Expand Down
174 changes: 21 additions & 153 deletions beartype/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -62,158 +62,26 @@
'''


# Intentionally defined last, as nobody wants to stumble into a full-bore rant
# first thing in the morning.
__all__ = ['STAR_IMPORTS_CONSIDERED_HARMFUL']
__all__ = ['beartype',]
'''
Special list global referencing a single attribute guaranteed *not* to exist.
The definition of this global effectively prohibits star imports from this
submodule into downstream modules by raising an :class:`AttributeError`
exception on the first attempt to do so: e.g.,
.. code-block:: shell-session
>>> from beartype import *
AttributeError: module 'beartype' has no attribute 'STAR_IMPORTS_CONSIDERED_HARMFUL'
All package submodules intentionally define similar ``__all__`` list globals.
Why? Because ``__all__`` is antithetical to sane API design and facilitates
antipatterns across the Python ecosystem, including well-known harms associated
with star imports and lesser-known harms associated with the artificial notion
of an ``__all__``-driven "virtual public API:" to wit,
* **Competing standards.** Thanks to ``__all__``, Python now provides two
conflicting conceptions of what constitutes the public API for a package or
module:
* The **conventional public API.** By convention, all module- and
class-scoped attributes *not* prefixed by ``_`` are public and thus
comprise the public API. Indeed, the standard interpretation of star
imports for packages and modules defining *no* ``__all__`` list globals is
exactly this.
* The **virtual public API.** By mandate, all module-scoped attributes
explicitly listed by the ``__all__`` global are public and thus *also*
comprise the public API. Consider the worst case of a module artificially
constructing this global to list all private module-scoped attributes
prefixed by ``_``; then the intersection of the conventional and virtual
public APIs for that module would be the empty list and these two competing
standards would be the list negations of one another. Ergo, the virtual
public API has no meaningful relation to the conventional public API or any
public attributes actually defined by any package or module.
These conflicting notions are evidenced no more strongly than throughout the
Python stdlib itself. Some stdlib modules ignore all notions of a public or
private API altogether (e.g., the :mod:`inspect` module, which
unconditionally introspects all attributes of various types regardless of
nomenclature or listing in ``__all__``); others respect only the conventional
public API (e.g., the :mod:`xmlrpc` package, whose server implementation
ignores ``_``-prefixed methods); still others respect only the virtual public
API (e.g., the :mod:`pickletools` module, which conditionally introspects the
:mod:`pickle` module via its ``__all__`` list global); still others respect
either depending on context with bizarre exceptions (e.g., the :mod:`pydoc`
module, which ignores attributes excluded from ``__all__`` for packages and
modules defining ``__all__`` and otherwise ignores ``_``-prefixed attributes
excluding :class:`collections.namedtuple` instances, which are considered
public because... reasons).
Which of these conflicted interpretations is correct? None and all of them,
since there is no correct interpretation. This is bad. This is even badder
for packages contractually adhering to `semver (i.e., semantic versioning)
<semver_>`__, despite there existing no uniform agreement in the Python
community as to what constitutes a "public Python API."
* **Turing completeness.** Technically, both the conventional and virtual
public APIs are defined only dynamically at runtime by the current
Turing-complete Python interpreter. Pragmatically, the conventional public
API is usually declared statically; those conventional public APIs that do
conditionally declare public attributes (e.g., to circumvent platform
portability concerns) often go to great and agonizing pains to declare a
uniform API with stubs raising exceptions on undefined edge cases. Deciding
the conventional public API for a package or module is thus usually trivial.
However, deciding the virtual public API for the same package or module is
often non-trivial or even infeasible. While many packages and modules
statically define ``__all__`` to be a simple context-independent list, others
dynamically append and extend ``__all__`` with context-dependent list
operations mystically depending on heterogeneous context *not* decidable at
authoring time -- including obscure incantations such as the active platform,
flags and options enabled at compilation time for the active Python
interpreter and C extensions, the conjunction of celestial bodies in
accordance with astrological scripture, and abject horrors like dynamically
extending the ``__all__`` exported from one submodule with the ``__all__``
exported from another not under the control of the author of the first.
(``__all__`` gonna ``__all__``, bro.)
* **Extrinsic cognitive load.** To decide what constitutes the "public API" for
any given package or module, rational decision-making humans supposedly
submit to a sadistic heuristic resembling the following:
* Does the package in question define a non-empty ``__init__`` submodule
defining a non-empty ``__all__`` list global? If so, the attributes listed
by this global comprise that package's public API. Since ``__all__`` is
defined only at runtime by a Turing-complete interpreter, however,
deciding these attributes is itself a Turing-complete problem.
* Else, all non-underscored attributes comprise that package's public API.
Since these attributes are also defined only at runtime by a
Turing-complete interpreter, deciding these attributes is again a
Turing-complete problem -- albeit typically less so. (See above.)
* **Redundancy.** ``__all__`` violates the DRY (Don't Repeat Yourself)
principle, thus inviting accidental desynchronization and omissions between
the conventional and virtual public APIs for a package or module. This leads
directly to...
* **Fragility.** By definition, accidentally excluding a public attribute from
the conventional public API is infeasible; either an attribute is public by
convention or it isn't. Conversely, accidentally omitting a public attribute
from the virtual public API is a trivial and all-too-common mishap. Numerous
stdlib packages and modules do so. This includes the pivotal :mod:`socket`
module, whose implementation in the Python 3.6.x series accidentally excludes
the public :func:`socket.socketpair` function from ``__all__`` if and only if
the private :mod:`_socket` C extension also defines the same function -- a
condition with no reasonable justification. Or *is* there? Dare mo shiranai.
* **Inconsistency.** Various modules and packages that declare ``__all__``
randomly exclude public attributes for spurious and frankly indefensible
reasons. This includes the stdlib :mod:`typing` module, whose preamble reads:
The pseudo-submodules 're' and 'io' are part of the public
namespace, but excluded from __all__ because they might stomp on
legitimate imports of those modules.
This is the worst of all possible worlds. A package or module either:
* Leave ``__all__`` undefined (as most packages and modules do).
* Prohibit ``__all__`` (as :mod:`beartype` does).
* Define ``__all__`` in a self-consistent manner conforming to
well-established semantics, conventions, and expectations.
* **Nonconsensus.** No consensus exists amongst either stdlib developers or the
Python community as a whole as to the interpretation of ``__all__``.
Third-party authors usually ignore ``__all__`` with respect to its role in
declaring a virtual public API, instead treating ``__all__`` as a means of
restricting star imports to some well-defined subset of public attributes.
This includes SciPy, whose :attr:`scipy.__init__.__all__` list global
excludes most public subpackages of interest (e.g., :mod:`scipy.linalg`, a
subpackage of linear algebra routines) while paradoxically including some
public subpackages of little to no interest (e.g., :attr:`scipy.test`, a unit
test scaffold commonly run only by developers and maintainers).
* **Infeasibility.** We have established that no two packages or modules
(including both stdlib and third-party) agree as to the usage of ``__all__``.
Respecting the virtual public API would require authors to ignore public
attributes omitted from ``__all__``, including those omitted by either
accident or due to conflicting interpretations of ``__all__``. Since this is
pragmatically infeasible *and* since upstream packages cannot reasonably
prohibit downstream packages from importing public attributes either
accidentally or intentionally excluded from ``__all__``, most authors
justifiably ignore ``__all__``. (*So should you.*)
* **Insufficiency.** The ``__all__`` list global only applies to module-scoped
attributes; there exists no comparable special attribute for classes with
which to define a comparable "virtual public class API." Whereas the
conventional public API uniformly applies to *all* attributes regardless of
scoping, the virtual public API only applies to module-scoped attributes -- a
narrower and less broadly applicable use case.
* **Unnecessity.** The conventional public API already exists. The virtual
public API offers no tangible benefits over the conventional public API while0
offering all the above harms. Under any rational assessment, the virtual
public API can only be "considered harmful."
.. _semver:
https://semver.org
Special list global of the unqualified names of all public package attributes
explicitly exported by and thus safely importable from this package.
Caveats
-------
**This global is defined only for conformance with static type checkers,** a
necessary prerequisite for `PEP 561`_-compliance. This global is *not* intended
to enable star imports of the form ``from beartype import *`` (now largely
considered a harmful anti-pattern by the Python community), although it
technically does the latter as well.
This global would ideally instead reference *only* a single package attribute
guaranteed *not* to exist (e.g., ``'STAR_IMPORTS_CONSIDERED_HARMFUL'``),
effectively disabling star imports. Since doing so induces spurious static
type-checking failures, we reluctantly embrace the standard approach. For
example, :mod:`mypy` emits an error resembling ``"error: Module 'beartype' does
not explicitly export attribute 'beartype'; implicit reexport disabled."``
.. _PEP 561:
https://www.python.org/dev/peps/pep-0561
'''
2 changes: 1 addition & 1 deletion beartype/_cave/abc.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@
from abc import ABCMeta, abstractmethod
from typing import Type

# See the "beartype.__init__" submodule for further commentary.
# See the "beartype.cave" submodule for further commentary.
__all__ = ['STAR_IMPORTS_CONSIDERED_HARMFUL']

# ....................{ FUNCTIONS }....................
Expand Down
2 changes: 1 addition & 1 deletion beartype/_cave/mapping.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
die_unless_hint_nonpep)
from typing import Union, Optional

# See the "beartype.__init__" submodule for further commentary.
# See the "beartype.cave" submodule for further commentary.
__all__ = ['STAR_IMPORTS_CONSIDERED_HARMFUL']

# ....................{ CONSTANTS }....................
Expand Down
2 changes: 1 addition & 1 deletion beartype/_decor/_cache/cachehint.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
from collections.abc import Callable
from typing import Any, Dict, Union

# See the "beartype.__init__" submodule for further commentary.
# See the "beartype.cave" submodule for further commentary.
__all__ = ['STAR_IMPORTS_CONSIDERED_HARMFUL']

# ....................{ GLOBALS }....................
Expand Down
2 changes: 1 addition & 1 deletion beartype/_decor/_cache/cachetype.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@
)
from typing import Tuple

# See the "beartype.__init__" submodule for further commentary.
# See the "beartype.cave" submodule for further commentary.
__all__ = ['STAR_IMPORTS_CONSIDERED_HARMFUL']

# ....................{ CONSTANTS }....................
Expand Down
2 changes: 1 addition & 1 deletion beartype/_decor/_code/_pep/_error/_peperrorgeneric.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@
from beartype._util.hint.pep.utilhintpeptest import is_hint_pep_typing
from typing import Generic, Optional

# See the "beartype.__init__" submodule for further commentary.
# See the "beartype.cave" submodule for further commentary.
__all__ = ['STAR_IMPORTS_CONSIDERED_HARMFUL']

# ....................{ GETTERS }....................
Expand Down
2 changes: 1 addition & 1 deletion beartype/_decor/_code/_pep/_error/_peperrorreturn.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
from beartype._decor._code._pep._error._peperrorsleuth import CauseSleuth
from beartype._util.text.utiltextlabel import label_callable

# See the "beartype.__init__" submodule for further commentary.
# See the "beartype.cave" submodule for further commentary.
__all__ = ['STAR_IMPORTS_CONSIDERED_HARMFUL']

# ....................{ GETTERS }....................
Expand Down
2 changes: 1 addition & 1 deletion beartype/_decor/_code/_pep/_error/_peperrorsequence.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
from beartype._util.text.utiltextrepr import get_object_representation
from typing import Optional

# See the "beartype.__init__" submodule for further commentary.
# See the "beartype.cave" submodule for further commentary.
__all__ = ['STAR_IMPORTS_CONSIDERED_HARMFUL']

# ....................{ GETTERS ~ sequence }....................
Expand Down
2 changes: 1 addition & 1 deletion beartype/_decor/_code/_pep/_error/_peperrorsleuth.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
)
from typing import Any, Callable, NoReturn, Optional, Tuple

# See the "beartype.__init__" submodule for further commentary.
# See the "beartype.cave" submodule for further commentary.
__all__ = ['STAR_IMPORTS_CONSIDERED_HARMFUL']

# ....................{ CLASSES }....................
Expand Down
2 changes: 1 addition & 1 deletion beartype/_decor/_code/_pep/_error/_peperrortype.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
from beartype._util.text.utiltextcause import get_cause_object_not_type
from typing import Optional

# See the "beartype.__init__" submodule for further commentary.
# See the "beartype.cave" submodule for further commentary.
__all__ = ['STAR_IMPORTS_CONSIDERED_HARMFUL']

# ....................{ GETTERS ~ forwardref }....................
Expand Down
2 changes: 1 addition & 1 deletion beartype/_decor/_code/_pep/_error/_peperrorunion.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
from beartype._util.text.utiltextrepr import get_object_representation
from typing import Optional

# See the "beartype.__init__" submodule for further commentary.
# See the "beartype.cave" submodule for further commentary.
__all__ = ['STAR_IMPORTS_CONSIDERED_HARMFUL']

# ....................{ GETTERS }....................
Expand Down
2 changes: 1 addition & 1 deletion beartype/_decor/_code/_pep/_error/peperror.py
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,7 @@
from collections.abc import Callable
from typing import Generic, Optional

# See the "beartype.__init__" submodule for further commentary.
# See the "beartype.cave" submodule for further commentary.
__all__ = ['STAR_IMPORTS_CONSIDERED_HARMFUL']

# ....................{ MAPPINGS }....................
Expand Down
2 changes: 1 addition & 1 deletion beartype/_decor/_code/_pep/_pephint.py
Original file line number Diff line number Diff line change
Expand Up @@ -1731,7 +1731,7 @@
from itertools import count
from typing import Set, Generic, Tuple, NoReturn, Optional

# See the "beartype.__init__" submodule for further commentary.
# See the "beartype.cave" submodule for further commentary.
__all__ = ['STAR_IMPORTS_CONSIDERED_HARMFUL']

# ....................{ CONSTANTS ~ hint : meta }....................
Expand Down
2 changes: 1 addition & 1 deletion beartype/_decor/_code/_pep/pepcode.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@
from inspect import Parameter
from typing import NoReturn, Tuple, Union

# See the "beartype.__init__" submodule for further commentary.
# See the "beartype.cave" submodule for further commentary.
__all__ = ['STAR_IMPORTS_CONSIDERED_HARMFUL']

# ....................{ CONSTANTS }....................
Expand Down
2 changes: 1 addition & 1 deletion beartype/_decor/_code/codemain.py
Original file line number Diff line number Diff line change
Expand Up @@ -586,7 +586,7 @@
from inspect import Parameter, Signature
from typing import Tuple

# See the "beartype.__init__" submodule for further commentary.
# See the "beartype.cave" submodule for further commentary.
__all__ = ['STAR_IMPORTS_CONSIDERED_HARMFUL']

# ....................{ CONSTANTS ~ private }....................
Expand Down
2 changes: 1 addition & 1 deletion beartype/_decor/_data.py
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@
from inspect import Signature
from typing import Optional

# See the "beartype.__init__" submodule for further commentary.
# See the "beartype.cave" submodule for further commentary.
__all__ = ['STAR_IMPORTS_CONSIDERED_HARMFUL']

# ....................{ CLASSES }....................
Expand Down
2 changes: 1 addition & 1 deletion beartype/_decor/_pep563.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
from beartype._util.text.utiltextlabel import label_callable_decorated_pith
from sys import modules as sys_modules

# See the "beartype.__init__" submodule for further commentary.
# See the "beartype.cave" submodule for further commentary.
__all__ = ['STAR_IMPORTS_CONSIDERED_HARMFUL']

# ....................{ RESOLVERS }....................
Expand Down
12 changes: 6 additions & 6 deletions beartype/_decor/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -238,11 +238,11 @@
from beartype._decor._code._pep._error.peperror import (
raise_pep_call_exception)
from beartype._util.text.utiltextmunge import number_lines
from typing import TYPE_CHECKING
from typing import Callable, TYPE_CHECKING
# from beartype._util.utilobject import get_object_name
# from types import FunctionType

# See the "beartype.__init__" submodule for further commentary.
# See the "beartype.cave" submodule for further commentary.
__all__ = ['STAR_IMPORTS_CONSIDERED_HARMFUL']

# ....................{ CONSTANTS }....................
Expand Down Expand Up @@ -271,7 +271,7 @@
'''

# ....................{ DECORATORS }....................
def beartype(func):
def beartype(func: Callable) -> Callable:
'''
Decorate the passed **pure-Python callable** (e.g., function or method
declared in Python rather than C) to validate both all annotated parameters
Expand All @@ -292,14 +292,14 @@ def beartype(func):
Parameters
----------
func : CallableTypes
func : Callable
**Non-class callable** (i.e., callable object that is *not* a class) to
be decorated by a dynamically generated new callable wrapping this
original callable with pure-Python type-checking.
Returns
----------
CallableTypes
Callable
Dynamically generated new callable wrapping this original callable with
pure-Python type-checking.
Expand Down Expand Up @@ -590,7 +590,7 @@ def beartype(func):
# return
#
# Tragically, Python fails to support module-scoped "return" statements. *sigh*
def beartype(func):
def beartype(func: Callable) -> Callable:
'''
Identity decorator.
Expand Down
2 changes: 1 addition & 1 deletion beartype/_util/cache/pool/utilcachepoollistfixed.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
from beartype.roar import _BeartypeUtilCachedFixedListException
from collections.abc import Iterable, Sized

# See the "beartype.__init__" submodule for further commentary.
# See the "beartype.cave" submodule for further commentary.
__all__ = ['STAR_IMPORTS_CONSIDERED_HARMFUL']

# ....................{ CONSTANTS }....................
Expand Down

0 comments on commit 2c9c92f

Please sign in to comment.