[BUG?] Generic types at module scope #256

skeggse · 2023-07-19T19:15:43Z

lidatong/dataclasses-json#371 seems to have made beartype unhappy in my codebase that uses both dataclasses_json and beartype:

Traceback (most recent call last):
  File ".../python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File ".../python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File ".../path2.py", line 14, in <module>
    from <path1> import CustomDataclassType
  File ".../path1.py", line 66, in <module>
    class CustomDataclassType(JsonDataclass["CustomDataclassType"]):
  File "/usr/local/lib/python3.9/site-packages/beartype/_decor/_cache/cachedecor.py", line 77, in beartype
    return beartype_object(obj, conf)
  File "/usr/local/lib/python3.9/site-packages/beartype/_decor/decorcore.py", line 129, in beartype_object
    return _beartype_type(  # type: ignore[return-value]
  File "/usr/local/lib/python3.9/site-packages/beartype/_decor/decorcore.py", line 743, in _beartype_type
    attr_value_beartyped = beartype_object(
  File "/usr/local/lib/python3.9/site-packages/beartype/_decor/decorcore.py", line 170, in beartype_object
    return _beartype_decorator_builtin(  # type: ignore[return-value]
  File "/usr/local/lib/python3.9/site-packages/beartype/_decor/decorcore.py", line 427, in _beartype_decorator_builtin
    func_checked = _beartype_func(
  File "/usr/local/lib/python3.9/site-packages/beartype/_decor/decorcore.py", line 611, in _beartype_func
    func_wrapper_code = generate_code(bear_call)
  File "/usr/local/lib/python3.9/site-packages/beartype/_decor/_wrap/wrapmain.py", line 185, in generate_code
    code_check_return = _code_check_return(bear_call)
  File "/usr/local/lib/python3.9/site-packages/beartype/_decor/_wrap/wrapmain.py", line 667, in _code_check_return
    reraise_exception_placeholder(
  File "/usr/local/lib/python3.9/site-packages/beartype/_util/error/utilerror.py", line 212, in reraise_exception_placeholder
    raise exception.with_traceback(exception.__traceback__)
  File "/usr/local/lib/python3.9/site-packages/beartype/_decor/_wrap/wrapmain.py", line 625, in _code_check_return
    code_return_check_pith_unmemoized = _unmemoize_func_wrapper_code(
  File "/usr/local/lib/python3.9/site-packages/beartype/_decor/_wrap/wrapmain.py", line 763, in _unmemoize_func_wrapper_code
    new=register_typistry_forwardref(
  File "/usr/local/lib/python3.9/site-packages/beartype/_util/cache/utilcachecall.py", line 278, in _callable_cached
    raise exception
  File "/usr/local/lib/python3.9/site-packages/beartype/_util/cache/utilcachecall.py", line 270, in _callable_cached
    return_value = args_flat_to_return_value[args_flat] = func(
  File "/usr/local/lib/python3.9/site-packages/beartype/_decor/_cache/cachetype.py", line 81, in register_typistry_forwardref
    die_unless_module_attr_name(
  File "/usr/local/lib/python3.9/site-packages/beartype/_util/mod/utilmodtest.py", line 96, in die_unless_module_attr_name
    raise exception_cls(
beartype.roar.BeartypeDecorHintForwardRefException: Forward reference "dataclasses_json.api.SchemaType[A]" syntactically invalid as module attribute name.

$ pip show beartype | rg Version
Version: 0.14.1

This is interesting to me, because it sure seems like it's valid code, and it feels like a simple implementation gap, but I am lacking a bunch of context and knowledge to properly evaluate this.

Minimal repro:

from dataclasses import dataclass

from beartype import beartype
from dataclasses_json import dataclass_json


@beartype
@dataclass_json
@dataclass
class CustomDataclassType:
    pass

Now, I'm sure you'd prefer a minimal repro that doesn't involve third-party library, but I think it's worth seeing in context. However, here's a repro that doesn't depend on dataclasses_json, though it does so at the expense of being less terse :)

import abc
from typing import Type, TypeVar

from beartype import beartype

A = TypeVar("A", bound="Mixin")


class Mixin(abc.ABC):
    @classmethod
    def schema(cls: Type[A]) -> "SchemaType[A]":
        pass


def mix(cls):
    cls.schema = classmethod(Mixin.schema.__func__)
    Mixin.register(cls)
    return cls


@beartype
@mix
class Custom:
    pass

This produces an error that looks very similar to the earlier one:

/tmp/repro$ python main.py
Traceback (most recent call last):
  File "/private/tmp/repro/main.py", line 23, in <module>
    class Custom:
  File "/usr/local/lib/python3.9/site-packages/beartype/_decor/_cache/cachedecor.py", line 77, in beartype
    return beartype_object(obj, conf)
  File "/usr/local/lib/python3.9/site-packages/beartype/_decor/decorcore.py", line 129, in beartype_object
    return _beartype_type(  # type: ignore[return-value]
  File "/usr/local/lib/python3.9/site-packages/beartype/_decor/decorcore.py", line 743, in _beartype_type
    attr_value_beartyped = beartype_object(
  File "/usr/local/lib/python3.9/site-packages/beartype/_decor/decorcore.py", line 170, in beartype_object
    return _beartype_decorator_builtin(  # type: ignore[return-value]
  File "/usr/local/lib/python3.9/site-packages/beartype/_decor/decorcore.py", line 427, in _beartype_decorator_builtin
    func_checked = _beartype_func(
  File "/usr/local/lib/python3.9/site-packages/beartype/_decor/decorcore.py", line 611, in _beartype_func
    func_wrapper_code = generate_code(bear_call)
  File "/usr/local/lib/python3.9/site-packages/beartype/_decor/_wrap/wrapmain.py", line 185, in generate_code
    code_check_return = _code_check_return(bear_call)
  File "/usr/local/lib/python3.9/site-packages/beartype/_decor/_wrap/wrapmain.py", line 667, in _code_check_return
    reraise_exception_placeholder(
  File "/usr/local/lib/python3.9/site-packages/beartype/_util/error/utilerror.py", line 212, in reraise_exception_placeholder
    raise exception.with_traceback(exception.__traceback__)
  File "/usr/local/lib/python3.9/site-packages/beartype/_decor/_wrap/wrapmain.py", line 625, in _code_check_return
    code_return_check_pith_unmemoized = _unmemoize_func_wrapper_code(
  File "/usr/local/lib/python3.9/site-packages/beartype/_decor/_wrap/wrapmain.py", line 763, in _unmemoize_func_wrapper_code
    new=register_typistry_forwardref(
  File "/usr/local/lib/python3.9/site-packages/beartype/_util/cache/utilcachecall.py", line 278, in _callable_cached
    raise exception
  File "/usr/local/lib/python3.9/site-packages/beartype/_util/cache/utilcachecall.py", line 270, in _callable_cached
    return_value = args_flat_to_return_value[args_flat] = func(
  File "/usr/local/lib/python3.9/site-packages/beartype/_decor/_cache/cachetype.py", line 81, in register_typistry_forwardref
    die_unless_module_attr_name(
  File "/usr/local/lib/python3.9/site-packages/beartype/_util/mod/utilmodtest.py", line 96, in die_unless_module_attr_name
    raise exception_cls(
beartype.roar.BeartypeDecorHintForwardRefException: Forward reference "__main__.SchemaType[A]" syntactically invalid as module attribute name.

The text was updated successfully, but these errors were encountered:

leycec · 2023-07-19T19:32:41Z

Gah! Thanks so much for this wonderfully detailed issue, @skeggse. You are entirely correct about everything, which must be why you still devbro in Silicon Valley whereas I feebly fled to the Canadian wilderness. I regret nothing.

Sadly, this is technically a duplicate of feature request #226: [Feature Request] Postponed evaluation of non-self-referential type hints. Let's shift you over there and copy-and-paste your excellent minimal-reproducible example (MRE), which I promise to resolve "shortly." 😓

I've been delaying addressing this due to the beartype.claw API and its automagical import hooks, which will either redefine the face of Python as we know it or catastrophically blow up in my face. Both of those things might even happen.

skeggse · 2023-07-19T19:50:07Z

Aha! I evidently struggle to use Github's search tool to find what I'm looking for. Happily, your index is much better tuned on the relevant keywords than the amalgam that is me + Github 🥳

You are entirely correct about everything, which must be why you still devbro in Silicon Valley whereas I feebly fled to the Canadian wilderness.

I cannot attest to whether you "feebly fled" but I somehow doubt that a bit 😄 I still "devbro" in SV because I must be some sort of masochist.

I've been delaying addressing this due to the beartype.claw API and its automagical import hooks, which will either redefine the face of Python as we know it or catastrophically blow up in my face. Both of those things might even happen.

This link is maybe the wrong link? I don't see any references to beartype.claw on that issue.

Thank you for pointing out the error in my ways, at least. I'll have to ponder how to move forward here; being unable to upgrade another dependency is non-ideal.

leycec · 2023-07-19T21:03:28Z

I still "devbro" in SV because I must be some sort of masochist.

...heh. It's such an intense place on the one hand; it's the epicentre of world-straddling AI on the other hand. I suspect these two hands have something to do with one another. 😆

This link is maybe the wrong link? I don't see any references to beartype.claw on that issue.

Gah! That link was totally the wrong link. This is the right link. ^{Caution: things got pretty intense there.}

The tl;dr here is that @beartype 0.15.0 to be released this Friday ^{I swear this on my cats.} is introducing a new beartype.claw API for installing import hooks. beartype.claw elevates @beartype to the first usable hybrid runtime static type-checker. By using beartype.claw, @beartype will now perform both static type-checking and runtime type-checking at runtime. Pretend what I said just made sense.

One-liner or it didn't happen, so:

# In the "{your_package}.__init__" submodule:
from beartype.claw import beartype_this_package

# After this Friday, this is what *EVERYONE* should now do.
# Nobody needs to manually decorate anything by @beartype anymore.
# This isn't just syntactic sugar, though. Doing this also instructs
# @beartype to type-check annotated assignment statements previously
# only type-checked by static type-checkers like mypy and pyright: e.g.,
#      # @beartype will now type-check this and raise a violation at runtime.
#      not_an_int: int = 'You lie, not_an_int! Why do you lie so much?'
beartype_this_package()

Flex those bear muscles. 💪 🐻

leycec · 2023-07-19T21:08:55Z

Oh, and...

Thank you for pointing out the error in my ways, at least. I'll have to ponder how to move forward here; being unable to upgrade another dependency is non-ideal.

No, no! The error is entirely ours; thanks so much for reminding me of our unresolved breakage. I fully agree that @beartype's behaviour is total suckage here. I hadn't realized that third-party packages were beginning to annotate everything with non-standard forward references like 'SchemaType[A]', which then forces @beartype to adjust accordingly.

Sadly, the only sane path forward for you is probably to temporarily disable @beartype on @dataclass_json-decorated dataclasses. That's sucky for obvious reasons, but... Worky is better than breaky. Take all my salty tears and turn them into working commits.

leycec closed this as completed Jul 19, 2023

leycec mentioned this issue Jul 19, 2023

[Feature Request] Postponed evaluation of non-self-referential type hints #226

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BUG?] Generic types at module scope #256

[BUG?] Generic types at module scope #256

skeggse commented Jul 19, 2023

leycec commented Jul 19, 2023

skeggse commented Jul 19, 2023

leycec commented Jul 19, 2023

leycec commented Jul 19, 2023

[BUG?] Generic types at module scope #256

[BUG?] Generic types at module scope #256

Comments

skeggse commented Jul 19, 2023

leycec commented Jul 19, 2023

skeggse commented Jul 19, 2023

leycec commented Jul 19, 2023

leycec commented Jul 19, 2023