Use Python type hints

[osma]

Since Python 3.5 it has been possible to add type hints for variables and function/method signatures (PEP 484).

We should do this within the Annif codebase as well. Here is a good basic tutorial about the available types and how to use them. Quoting from the post:

Type hints bring an additional layer of abstraction on top of your code: they help to document it, clarify the assumptions about inputs/outputs and prevent insidious and silent mistakes when static code analysis (👋 mypy) is performed on top.

Eventually we should also enforce the type hints with mypy, but let's leave that out of scope for this issue so that it can be closed once we have the hints in place at least for a reasonable proportion of the codebase.

[juhoinkinen]

A few tools have been mentioned that can help by (semi)automatically generating the type annotations:

Sure, there are some projects out there that can help out.

Instagram developed the MonkeyType tool to pick up runtime type information. These are output as stubs files, but you can use the tools included to apply those to your source files as inline annotations, or use the retype project to do the same.
Dropbox has a similar tool called pyannotate; this tool can generate either Python 2 type hint comments or annotations.

Neither project will produce perfect type hints; always take their output as a starting point, not the authoritative final annotations.

[juhoinkinen]

There is also a pytest-monkeytype plugin that can be used with monkeytype for recording the variable types.

[juhoinkinen]

As instructed, I run py.test --monkeytype-output=./monkeytype.sqlite3 to trace and record the variable types to/from functions based on pytest run. This data covers also the unit tests.

Then I run monkeytype apply <module> --pep_563 to all modules (not unit tests) to insert the type annotations to function signatures. The --pep_563 option makes monkeytype to insert also from __future__ import annotations to altered files (needed in Python <3.11) and to wrap imports needed for signatures in if TYPE_CHECKING: conditional blocks.

PEP563 is for solving two problems:

• forward references: when a type hint contains names that have not been defined yet, that definition needs to be expressed as a string literal;
• type hints are executed at module import time, which is not computationally free.

These changes by monkeytype can be used as a starting point for manual editing.