Permalink
604 lines (429 sloc) 18.6 KB

Common issues and solutions

This section has examples of cases when you need to update your code to use static typing, and ideas for working around issues if mypy doesn't work as expected. Statically typed code is often identical to normal Python code (except for type annotations), but sometimes you need to do things slightly differently.

Can't install mypy using pip

If installation fails, you've probably hit one of these issues:

  • Mypy needs Python 3.4 or later to run.
  • You may have to run pip like this: python3 -m pip install mypy.

No errors reported for obviously wrong code

There are several common reasons why obviously wrong code is not flagged as an error.

  • The function containing the error is not annotated. Functions that do not have any annotations (neither for any argument nor for the return type) are not type-checked, and even the most blatant type errors (e.g. 2 + 'a') pass silently. The solution is to add annotations. Where that isn't possible, functions without annotations can be checked using --check-untyped-defs.

    Example:

    def foo(a):
        return '(' + a.split() + ')'  # No error!

    This gives no error even though a.split() is "obviously" a list (the author probably meant a.strip()). The error is reported once you add annotations:

    def foo(a: str) -> str:
        return '(' + a.split() + ')'
    # error: Unsupported operand types for + ("str" and List[str])

    If you don't know what types to add, you can use Any, but beware:

  • One of the values involved has type 'Any'. Extending the above example, if we were to leave out the annotation for a, we'd get no error:

    def foo(a) -> str:
        return '(' + a.split() + ')'  # No error!

    The reason is that if the type of a is unknown, the type of a.split() is also unknown, so it is inferred as having type Any, and it is no error to add a string to an Any.

    If you're having trouble debugging such situations, :ref:`reveal_type() <reveal-type>` might come in handy.

    Note that sometimes library stubs have imprecise type information, e.g. the pow() builtin returns Any (see typeshed issue 285 for the reason).

  • Some imports may be silently ignored. Another source of unexpected Any values are the :ref:`"--ignore-missing-imports" <ignore-missing-imports>` and :ref:`"--follow-imports=skip" <follow-imports>` flags. When you use --ignore-missing-imports, any imported module that cannot be found is silently replaced with Any. When using --follow-imports=skip the same is true for modules for which a .py file is found but that are not specified on the command line. (If a .pyi stub is found it is always processed normally, regardless of the value of --follow-imports.) To help debug the former situation (no module found at all) leave out --ignore-missing-imports; to get clarity about the latter use --follow-imports=error. You can read up about these and other useful flags in :ref:`command-line`.

  • A function annotated as returning a non-optional type returns 'None' and mypy doesn't complain.

    def foo() -> str:
        return None  # No error!

    You may have disabled strict optional checking (see :ref:`no_strict_optional` for more).

Spurious errors and locally silencing the checker

You can use a # type: ignore comment to silence the type checker on a particular line. For example, let's say our code is using the C extension module frobnicate, and there's no stub available. Mypy will complain about this, as it has no information about the module:

import frobnicate  # Error: No module "frobnicate"
frobnicate.start()

You can add a # type: ignore comment to tell mypy to ignore this error:

import frobnicate  # type: ignore
frobnicate.start()  # Okay!

The second line is now fine, since the ignore comment causes the name frobnicate to get an implicit Any type.

Note

The # type: ignore comment will only assign the implicit Any type if mypy cannot find information about that particular module. So, if we did have a stub available for frobnicate then mypy would ignore the # type: ignore comment and typecheck the stub as usual.

Another option is to explicitly annotate values with type Any -- mypy will let you perform arbitrary operations on Any values. Sometimes there is no more precise type you can use for a particular value, especially if you use dynamic Python features such as __getattr__:

class Wrapper:
    ...
    def __getattr__(self, a: str) -> Any:
        return getattr(self._wrapped, a)

Finally, you can create a stub file (.pyi) for a file that generates spurious errors. Mypy will only look at the stub file and ignore the implementation, since stub files take precedence over .py files.

Unexpected errors about 'None' and/or 'Optional' types

Starting from mypy 0.600, mypy uses :ref:`strict optional checking <strict_optional>` by default, and the None value is not compatible with non-optional types. It's easy to switch back to the older behavior where None was compatible with arbitrary types (see :ref:`no_strict_optional`). You can also fall back to this behavior if strict optional checking would require a large number of assert foo is not None checks to be inserted, and you want to minimize the number of code changes required to get a clean mypy run.

Mypy runs are slow

If your mypy runs feel slow, you should probably use the :ref:`mypy daemon <mypy_daemon>`, which can speed up incremental mypy runtimes by a factor of 10 or more. :ref:`Remote caching <remote-cache>` can make cold mypy runs several times faster.

Types of empty collections

You often need to specify the type when you assign an empty list or dict to a new variable, as mentioned earlier:

a: List[int] = []

Without the annotation mypy can't always figure out the precise type of a.

You can use a simple empty list literal in a dynamically typed function (as the type of a would be implicitly Any and need not be inferred), if type of the variable has been declared or inferred before, or if you perform a simple modification operation in the same scope (such as append for a list):

a = []  # Okay because followed by append, inferred type List[int]
for i in range(n):
    a.append(i * i)

However, in more complex cases an explicit type annotation can be required (mypy will tell you this). Often the annotation can make your code easier to understand, so it doesn't only help mypy but everybody who is reading the code!

Redefinitions with incompatible types

Each name within a function only has a single 'declared' type. You can reuse for loop indices etc., but if you want to use a variable with multiple types within a single function, you may need to declare it with the Any type.

def f() -> None:
    n = 1
    ...
    n = 'x'        # Type error: n has type int

Note

This limitation could be lifted in a future mypy release.

Note that you can redefine a variable with a more precise or a more concrete type. For example, you can redefine a sequence (which does not support sort()) as a list and sort it in-place:

def f(x: Sequence[int]) -> None:
    # Type of x is Sequence[int] here; we don't know the concrete type.
    x = list(x)
    # Type of x is List[int] here.
    x.sort()  # Okay!

Invariance vs covariance

Most mutable generic collections are invariant, and mypy considers all user-defined generic classes invariant by default (see :ref:`variance-of-generics` for motivation). This could lead to some unexpected errors when combined with type inference. For example:

class A: ...
class B(A): ...

lst = [A(), A()]  # Inferred type is List[A]
new_lst = [B(), B()]  # inferred type is List[B]
lst = new_lst  # mypy will complain about this, because List is invariant

Possible strategies in such situations are:

  • Use an explicit type annotation:

    new_lst: List[A] = [B(), B()]
    lst = new_lst  # OK
  • Make a copy of the right hand side:

    lst = list(new_lst) # Also OK
  • Use immutable collections as annotations whenever possible:

    def f_bad(x: List[A]) -> A:
        return x[0]
    f_bad(new_lst) # Fails
    
    def f_good(x: Sequence[A]) -> A:
        return x[0]
    f_good(new_lst) # OK

Declaring a supertype as variable type

Sometimes the inferred type is a subtype (subclass) of the desired type. The type inference uses the first assignment to infer the type of a name (assume here that Shape is the base class of both Circle and Triangle):

shape = Circle()    # Infer shape to be Circle
...
shape = Triangle()  # Type error: Triangle is not a Circle

You can just give an explicit type for the variable in cases such the above example:

shape = Circle() # type: Shape   # The variable s can be any Shape,
                                 # not just Circle
...
shape = Triangle()               # OK

Complex type tests

Mypy can usually infer the types correctly when using isinstance() type tests, but for other kinds of checks you may need to add an explicit type cast:

def f(o: object) -> None:
    if type(o) is int:
        o = cast(int, o)
        g(o + 1)    # This would be an error without the cast
        ...
    else:
        ...

Note

Note that the object type used in the above example is similar to Object in Java: it only supports operations defined for all objects, such as equality and isinstance(). The type Any, in contrast, supports all operations, even if they may fail at runtime. The cast above would have been unnecessary if the type of o was Any.

Mypy can't infer the type of o after the type() check because it only knows about isinstance() (and the latter is better style anyway). We can write the above code without a cast by using isinstance():

def f(o: object) -> None:
    if isinstance(o, int):  # Mypy understands isinstance checks
        g(o + 1)        # Okay; type of o is inferred as int here
        ...

Type inference in mypy is designed to work well in common cases, to be predictable and to let the type checker give useful error messages. More powerful type inference strategies often have complex and difficult-to-predict failure modes and could result in very confusing error messages. The tradeoff is that you as a programmer sometimes have to give the type checker a little help.

Python version and system platform checks

Mypy supports the ability to perform Python version checks and platform checks (e.g. Windows vs Posix), ignoring code paths that won't be run on the targeted Python version or platform. This allows you to more effectively typecheck code that supports multiple versions of Python or multiple operating systems.

More specifically, mypy will understand the use of sys.version_info and sys.platform checks within if/elif/else statements. For example:

import sys

# Distinguishing between different versions of Python:
if sys.version_info >= (3, 5):
    # Python 3.5+ specific definitions and imports
elif sys.version_info[0] >= 3:
    # Python 3 specific definitions and imports
else:
    # Python 2 specific definitions and imports

# Distinguishing between different operating systems:
if sys.platform.startswith("linux"):
    # Linux-specific code
elif sys.platform == "darwin":
    # Mac-specific code
elif sys.platform == "win32":
    # Windows-specific code
else:
    # Other systems

Note

Mypy currently does not support more complex checks, and does not assign any special meaning when assigning a sys.version_info or sys.platform check to a variable. This may change in future versions of mypy.

By default, mypy will use your current version of Python and your current operating system as default values for sys.version_info and sys.platform.

To target a different Python version, use the --python-version X.Y flag. For example, to verify your code typechecks if were run using Python 2, pass in --python-version 2.7 from the command line. Note that you do not need to have Python 2.7 installed to perform this check.

To target a different operating system, use the --platform PLATFORM flag. For example, to verify your code typechecks if it were run in Windows, pass in --platform win32. See the documentation for sys.platform for examples of valid platform parameters.

Displaying the type of an expression

You can use reveal_type(expr) to ask mypy to display the inferred static type of an expression. This can be useful when you don't quite understand how mypy handles a particular piece of code. Example:

reveal_type((1, 'hello'))  # Revealed type is 'Tuple[builtins.int, builtins.str]'

You can also use reveal_locals() at any line in a file to see the types of all local variables at once. Example:

a = 1
b = 'one'
reveal_locals()
# Revealed local types are:
# a: builtins.int
# b: builtins.str

Note

reveal_type and reveal_locals are only understood by mypy and don't exist in Python. If you try to run your program, you'll have to remove any reveal_type and reveal_locals calls before you can run your code. Both are always available and you don't need to import them.

Import cycles

An import cycle occurs where module A imports module B and module B imports module A (perhaps indirectly, e.g. A -> B -> C -> A). Sometimes in order to add type annotations you have to add extra imports to a module and those imports cause cycles that didn't exist before. If those cycles become a problem when running your program, there's a trick: if the import is only needed for type annotations in forward references (string literals) or comments, you can write the imports inside if TYPE_CHECKING: so that they are not executed at runtime. Example:

File foo.py:

from typing import List, TYPE_CHECKING

if TYPE_CHECKING:
    import bar

def listify(arg: 'bar.BarClass') -> 'List[bar.BarClass]':
    return [arg]

File bar.py:

from typing import List
from foo import listify

class BarClass:
    def listifyme(self) -> 'List[BarClass]':
        return listify(self)

Note

The TYPE_CHECKING constant defined by the typing module is False at runtime but True while type checking.

Python 3.5.1 doesn't have typing.TYPE_CHECKING. An alternative is to define a constant named MYPY that has the value False at runtime. Mypy considers it to be True when type checking. Here's the above example modified to use MYPY:

from typing import List

MYPY = False
if MYPY:
    import bar

def listify(arg: 'bar.BarClass') -> 'List[bar.BarClass]':
    return [arg]

Silencing linters

In some cases, linters will complain about unused imports or code. In these cases, you can silence them with a comment after type comments, or on the same line as the import:

# to silence complaints about unused imports
from typing import List  # noqa
a = None  # type: List[int]

To silence the linter on the same line as a type comment put the linter comment after the type comment:

a = some_complex_thing()  # type: ignore  # noqa

Covariant subtyping of mutable protocol members is rejected

Mypy rejects this because this is potentially unsafe. Consider this example:

from typing_extensions import Protocol

class P(Protocol):
    x: float

def fun(arg: P) -> None:
    arg.x = 3.14

class C:
    x = 42
c = C()
fun(c)  # This is not safe
c.x << 5  # Since this will fail!

To work around this problem consider whether "mutating" is actually part of a protocol. If not, then one can use a @property in the protocol definition:

from typing_extensions import Protocol

class P(Protocol):
    @property
    def x(self) -> float:
       pass

def fun(arg: P) -> None:
    ...

class C:
    x = 42
fun(C())  # OK

Dealing with conflicting names

Suppose you have a class with a method whose name is the same as an imported (or built-in) type, and you want to use the type in another method signature. E.g.:

class Message:
    def bytes(self):
        ...
    def register(self, path: bytes):  # error: Invalid type "mod.Message.bytes"
        ...

The third line elicits an error because mypy sees the argument type bytes as a reference to the method by that name. Other than renaming the method, a work-around is to use an alias:

bytes_ = bytes
class Message:
    def bytes(self):
        ...
    def register(self, path: bytes_):
        ...

I need a mypy bug fix that hasn't been released yet

You can install the latest development version of mypy from source. Clone the mypy repository on GitHub, and then run pip install locally:

git clone --recurse-submodules https://github.com/python/mypy.git
cd mypy
sudo python3 -m pip install --upgrade .