On Typing

Introduction

mypy is a static type checker for Python programming language. With mypy, developers can write more reliable and maintainable code, which reduces the likelihood of errors and bugs in their applications. mypy allows developers to specify the types of variables, function arguments, and return values in their code, which can help catch type-related errors during the development process.

Setup

Considering the setup done with How to develop in SuperDuperDB, one can run mypy inside the poetry virtual environment by running:

mypy

It is also possible to run mypy to verify a single file with:

mypy superduperdb/core/base.py

Configuration

The predefined configuration of the project is stored in mypy.ini and it is automatically used by mypy to run the analysis.

Configuration file

mypy.ini file follows the structure of Documentation of the configuration file.

Configuration Rules

There are several rules that can be used in the configuration file to define the level of strictness when analyzing the project. The list of strict rules are available in Introduce stricter options.

typing is hard

So we should just “do a good job”, not a perfect job.

It is currently effectively impossible to fully define even JSON using typing: see this discussion.

If we can’t even completely define perhaps the single most used type in programming, then it should be clear that perfect typing is impossible, so we should just do a good job.

Our goals

Our goals for typing are, in this order of priority:

1. Satisfying mypy
2. Helpfully documenting our API interfaces
3. Having more specific types

1 is a priority because it’s mechanical, and it’s a low bar.

Sometimes the most specific type is hard to write, but more, hard to understand, and since our primary goal is documentary, it might be better to have a somewhat less accurate type and use the time saved to write useful doc strings.

As a concrete example, t.Dict as a type is probably a cop-out, but t.Dict[str, t.Any] is significantly better, and t.Dict[str, SpecificType] would be better yet again, but in some cases, figuring out what SpecificType was might be more trouble than it is worth.

(See Conventions.)

No typing inside executable code

Because these types cannot be read by documentation engines which only import and do not parse the code.

Members should be typed early on the class definition whether or not the class is a data class.

Attributes will eventually require typing

Creating annotations might not seem important when creating attributes of an object, but we can easily run into an issue where there is a need to add annotations to a variable:

class BankAccount:
    def __init__(self):
        self.balance = 0

    def deposit(self, amount):
        self.balance += amount

    def withdraw(self, amount):
        self.balance -= amount

account = BankAccount()
account.deposit(100)
account.withdraw(50)
account.balance = "1000"

In this case mypy cannot infer the type of balance and cannot verify any faults with this code. It is then important to consider a small change:

class BankAccount:
    balance: int  
  
		def __init__(self):
        self.balance = 0

    def deposit(self, amount):
        self.balance += amount

    def withdraw(self, amount):
        self.balance -= amount

account = BankAccount()
account.deposit(100)
account.withdraw(50)
account.balance = "1000"

In this case mypy will identify the simple error: Incompatible types in assignment (expression has type "str", variable has type "int")