Automated type hint checking

[amblina]

I was looking to assist with the type checking of this project and support others adding type hints (#938). In order to do this effectively I suggest a github action is setup to do automatic testing. The current project has a significant number of non-trivial errors even when strict mode is disabled so outside of inline or module level overrides this stage would not pass without significant work. Adding CI would also lower the barrier of entry for contribution from new contributors. If this project were using GitLab CI I would suggest an allow_failure tag but this isn't available for Github afaik.

A few options in order to effort:

• Run mypy in CI but || true - this will not pick up regressions but will preserve green build.
• Run mypy in CI with || true but use an external library such as https://github.com/orsinium-labs/mypy-baseline to raise if new errors are introduced - preserve green build but only if no regressions found.
• Do anything required including in-line/module level overrides until the project is green and proceed from there - likely messy and will not highlight regressions if they have been ignored on the module level etc but will preserve green build

I'm happy to assist where I can with implementation and open to any other suggestions. In the end, knowing which hints can and can't be relied on will help with readability and maintainability in the long run 💪 .

[stevepiercy]

Thanks for opening this discussion. I think it's worth exploring.

GitHub Actions does not support allow_failure. See actions/runner#2347.

Another option is to document how to use the tools locally to identify typing issues. That would include working on a single file or Python object, instead of the entire code base. In any case, if CI fails, we need to tell the contributor how to fix the typing issues, so documentation would be required.

[niccokunzmann]

Hi,

thank you for your suggestion. It is great that you are passionate about type hints and I am happy that the project has progressed to a point where we can do the checks.
We can use the the same way we were working on the code formatting.

1. Implement automated tests that do not fail but highlight the issues.
2. Document and implement how to run the type checks on your computer.
3. Create a first-timer issue with all the files concerned. This allows everyone to chip in and solve the issues.
4. Once all issues are solved, we turn the type check into a mandatory test.
5. We can follow the same process after this for strict type checking or just do strict typing from the start - your decision.

That might take a few months until it is complete but we will get there.
I welcome your PR and drive to work on it. I will have time in 2 weeks to chip in.

If you want to contribute by monitoring PRs in steps 2-5, that would be much appreciated.