[py] Run type checks in CI and report typing coverage to codecov

[hoefling]

Description

This PR adds another Github action that runs type checks (using the mypy toxenv) and reports the current status to https://codecov.io. An example report, generated from this branch in my fork, can be viewed here: in the typing context, 100% coverage means all types could be successfully resolved and verified, while <100% indicates missing, incomplete or erroneous type hints. Navigating to a module unveils the spots in question.

Example: In the above example, the selenium.common package is fully typed (green color & 100% coverage), while selenium.webdriver.chrome.options module is not - more details can be found in tox_mypy job log, where mypy will report an error for the line. In this example:

selenium/webdriver/chrome/options.py:25: error: Implicit generic "Any". Use "typing.Dict" and specify generic parameters
selenium/webdriver/chrome/options.py:28: error: Function is missing a type annotation

(relevant spot in the job log for reference)

This way, one should be able to easily track the typing errors.

Motivation and Context

This is a proposal to address the discussion in #9482. It is added for @AutomatedTester for an evaluation!

Also, once merged into master, codecov should start tracking the changes in upcoming PRs, comparing typing coverage against the current master (an example on how it will look like, although this comment actually reports code coverage difference).

Once all typing errors are resolved (thus a 100% typing coverage achieved for the complete Python codebase), we can stop reporting the typing coverage and just run tox -c py/tox.ini in tox_mypy job. It will then simply fail on new typing errors.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)

Checklist

• I have read the contributing document.
• My change requires a change to the documentation.
• I have updated the documentation accordingly.
• I have added tests to cover my changes.
• All new and existing tests passed.

[hoefling]

Here's the initial report for the upstream repo. Of course, there's nothing on master to compare against yet, so the codecov-bot will leave no comments as of now.

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information

[diemol]

Is there a way to this by using SonarQube? We already use it for the Java bindings and probably keeping an eye on different places is not ideal.

[hoefling]

@diemol sure, but AFAIK SonarQube doesn't support mypy reports, only code coverage reports. Do you have plans to track test coverage with SonarQube?

[codecov-commenter]

Codecov Report

❗ No coverage uploaded for pull request base (trunk@e2866fc). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff            @@
##             trunk    #9523   +/-   ##
========================================
  Coverage         ?   44.09%           
========================================
  Files            ?       82           
  Lines            ?     5445           
  Branches         ?      247           
========================================
  Hits             ?     2401           
  Misses           ?     2797           
  Partials         ?      247           

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e2866fc...c0d3414. Read the comment docs.

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
No Duplication information