Prefer the mypy restrictions on type variables

[gvanrossum]

the PEP has this example:

X = Var('X')
Y = Var('X')
  def first(l: Sequence[X]) -> Y:   # Generic function
      return l[0]

In mypy, the equivalent thing with typevar() won't work -- Y = typevar('X') is rejected because the variable name is not the same as the name argument, and a type variable imported from another module with the same name still doesn't make the example work.

I like the mypy restrictions better.

See also python/mypy#539.

[ambv]

The core issue here is that we're trying to create an add-on to the language without modifying its syntax. Having X = TypeVar('X') work but not 'Y = TypeVar('X')` is confusing. Seems like we'd really want TypeVar to be a statement in the local scope. We could make it a "statement", e.g.

Var('X')
Var('AnyStr', bytes, str)

However, it would need to inject the name into the local scope which is just as ugly, I guess.

By the way, why does importing type variables not work?

[gvanrossum]

I propose to use mypy's rules. Every TypeVar() call creates a unique type variable. The static checker enforces that TypeVar() must always be assigned to a local (or global, depending on the current scope) variable whose name is the same as the string passed as argument. The runtime doesn't enforce this, but it still treats each TypeVar() as a unique type. For example:

T = TypeVar('T')
X = TypeVar('T')  # Fails static check, but OK at runtime

Now the following asserts all pass:

assert issubclass(T, T)
assert issubclass(X, X)
assert not issubclass(T, X)
assert not issubclass(X, T)
assert T != X

(Note that the first two simply check that issubclass of a type with itself should always be true.)

[gvanrossum]

Oh, my comment about import was simply that mypy will (of course) let you define type vars with the same name in multiple modules, and you can import them, but they will still be considered different. E.g. this passes in mypy:

from typing import AnyStr
def make_url(host: AnyStr, path: AnyStr) -> AnyStr:
    if isinstance(host, str):
        return 'https://' + host + path
    if isinstance(host, bytes):
        return b'https://' + host + path
print(make_url('www.python.org', '/dev/peps/'))

I can also systematically substitute typing.AnyStr for AnyStr.

It also works if I replace the import with:

import typing
AnyStr = typing.typevar('AnyStr', values=(str, bytes))

But after that if fails if I replace the return type with typing.AnyStr:

import typing
AnyStr = typing.typevar('AnyStr', values=(str, bytes))
def make_url(host: AnyStr, path: AnyStr) -> typing.AnyStr:
    if isinstance(host, str):
        return 'https://' + host + path
    if isinstance(host, bytes):
        return b'https://' + host + path
print(make_url('www.python.org', '/dev/peps/'))

The errors are:

z1.py: In function "make_url":
z1.py, line 5: Incompatible return value type: expected builtins.bytes*, got builtins.str
z1.py, line 7: Incompatible return value type: expected builtins.str*, got builtins.bytes

[ambv]

OK, I understand. Will update the PEP accordingly.

[JukkaL]

I updated the PEP at PyCon to describe the new type variable semantics. I think that this can be closed now.