Support recursive types

[o11c]

The following in particular would be useful:

Callback = Callable[[str], 'Callback']
Foo = Union[str, List['Foo']]

[JukkaL]

If (or when) mypy will have structural subtyping, recursive types would also be useful there.

[JukkaL]

My current plan is to postpone recursive types until simple structural subtyping is in and reconsider them later. After thinking about them more they are going to increase complexity a lot of I haven't seen much evidence for them being needed that often.

[oconnor663]

If I'm understanding this issue right, I'm running into it for classes that want a fluent interface. So for example, if I want callers to do something like this:

myfoo.add(bar).add(baz).finish()

Then the definition of the Foo class and the add method need to look something like this:

class Foo:
    def add(self, x) -> Foo:  # Python chokes on this line!
        # do stuff
        return self

Another place where Python commonly does return self is in the __enter__ method for context managers. Is mypy able to typecheck those right now?

[refi64]

@oconnor663 Try:

class Foo:
    def add(self, x) -> 'Foo':
        # do stuff
        return self

[oconnor663]

Ah, thank you.

[oconnor663]

@kirbyfan64, do you know if there are standard functions anywhere that understand this convention? Like, if I wanted to introspect a couple functions and compare the types of their arguments, should I handle the Foo == "Foo" case explicitly? That seems doable, but a string-aware version of say isinstance seems harder.

[refi64]

@oconnor663 I don't think there's anything like that. If you're introspecting the functions via a decorator, you could try accessing the caller's globals and locals.

[gvanrossum]

You're aware of typing.get_type_hints(obj)right? It is similar to obj.annotations` but expands forward references.
https://docs.python.org/3/library/typing.html?highlight=typing#typing.get_type_hints

There used to be an instance() implementation in typing.py but Mark Shannon
made me take it out. It's being deleted in this rev:
python/typing@ac7494f

On Fri, Oct 16, 2015 at 9:37 AM, Ryan Gonzalez notifications@github.com
wrote:

@oconnor663 https://github.com/oconnor663 I don't think there's
anything like that. If you're introspecting the functions via a decorator,
you could try accessing the caller's globals and locals.

—
Reply to this email directly or view it on GitHub
#731 (comment).

--Guido van Rossum (python.org/~guido)

[oconnor663]

@gvanrossum that's exactly what I was looking for, thanks. Sorry for the n00b questions today, but awesome that all this is supported.

[JukkaL]

Mypy should detect missing string literal escapes (see #948).

[dmoisset]

Going back to the original point on the issue, I found a case in the stdlib where this would be needed; the type for isinstance() is currently:

def isinstance(o: object, t: Union[type, Tuple[type, ...]]) -> bool: ...

but it should actually be:

ClassInfo = Union[type, Tuple['ClassInfo', ...]]
def isinstance(o: object, t: ClassInfo) -> bool: ...

Because according to https://docs.python.org/3/library/functions.html#isinstance the tuples can be nested. I found an actual example of this while typechecking django.http.response.HttpResponse.content

[srittau]

I have come across this while trying to define a generic JSON type:

JSON = Union[Dict[str, "JSON"], List["JSON"], str, int, float, bool, None]

So consider this a +1 for supporting this use case.

[graingert]

@srittau JSON needs to be Any because it is recursive and you can give json.loads a custom JSONEncoder class:

_PlainJSON = Union[Dict[str, "_PlainJSON"], List["_PlainJSON"], str, int, float, bool, None]
_T = TypeVar('_T')
JSON = Union[_PlainJSON, _T, Dict[str, "JSON"], List["JSON"]]
def loads(data: str, cls: Type[JSONEncoder[_T]]) -> JSON: ...

of course recursive types and Type[JSONEncoder[_T]] types arn't supported.

[DustinWehr]

The following pattern seems to be good enough for my purposes. The boilerplate is tolerable for me.

class BTree(NamedTuple):
    val: int
    left_ : Any
    right_ : Any

    # typed constructor
    @staticmethod
    def c(val: int, left: Optional['BTree'] = None, right: Optional['BTree'] = None) -> 'BTree':
        return BTree(val, left, right)

    # typed accessors
    @property
    def left(self) -> Optional['BTree']:
        return cast(Optional[BTree], self.left_)
    @property
    def right(self) -> Optional['BTree']:
        return cast(Optional[BTree], self.right_)

atree = BTree.c(1, BTree.c(2, BTree.c(3), BTree.c(4)), BTree.c(5))
atree2 = BTree.c(1, BTree.c(2, BTree.c(3), BTree.c(4)), BTree.c(5))
assert atree == atree2
assert isinstance(atree,BTree) and isinstance(atree.left,BTree) and isinstance(atree.left.left,BTree)

[DustinWehr]

Latest version of pattern. We use this example at Legalese for interacting with SMT solvers (the SMTLIB language).

I found that I ended up forgetting to use the typed .c static method instead of the untyped constructor in the BTree example above. This version addresses that. It's only very minor deficits are:

1. Boilerplate (which I don't care about if the datatype is significant enough for us to care about the difference between an immutable recursive datatype and Tuple[Any,...])
2. The constructor SMTExprNonatom and the type SMTExprNonatom_ do not share the same name. But this is only aesthetic; You won't use one when you mean the other, since with this naming convention, SMTExprNonatom will come up first in autocomplete, and only SMTExprNonatom_ can be used in a type position.

# Immutable recursive datatypes pattern
SMTAtom = Union[str, int, float, bool]
SMTExpr = Union['SMTExprNonatom_',SMTAtom]
class SMTExprNonatom_(NamedTuple):  
    symb: str
    args_: Tuple[Any,...] # see `args` below
    @staticmethod  # typed constructor, which we alias to `SMTExprNonatom` after the class def
    def c(symb:str, args:Iterable[SMTExpr]) -> 'SMTExprNonatom_': return SMTExprNonatom_(symb, tuple(args))
    @property # typed accessor
    def args(self) -> Tuple[SMTExpr]: return cast(Tuple[SMTExpr], self.args_)
SMTExprNonatom = SMTExprNonatom_.c
SMTCommand = NewType('SMTCommand', SMTExprNonatom_)

[cj81499]

@ilevkivskyi is there a way to set this config option in a pyproject.toml file? I checked the documentation, but couldn't find anything about the new cli flag (perhaps because it's still experimental, and will eventually be enabled by default?)

[ilevkivskyi]

Yes, it is indeed a temporary flag (for few releases), and in 0.990 it will be on by default, you can try

$ cat pyproject.toml
[tool.mypy]
enable_recursive_aliases = true

This worked for me on Linux.

[bartfeenstra]

It appears that v0.990/v0.991 may not have addressed the entirety of the challenge that is recursive/cyclic types: #14219

[jamesbraza]

For those reading in posterity:

1. mypy==0.981 added --enable-recursive-alias: Enable recursive type aliases behind a flag #13297
2. mypy==0.990 started a deprecation cycle for --enable-recursive-alias: Temporarily put back --enable-recursive-aliases (as depreceated) #13852
3. mypy==1.7.0 completed the deprecation cycle, removing --enable-recursive-alias: Delete recursive aliases flags #16346

So with mypy>=1.7, recursive type support is built into mypy ✅