Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add type checking (mypy) #1244

Merged
merged 7 commits into from Jun 30, 2023
Merged

Add type checking (mypy) #1244

merged 7 commits into from Jun 30, 2023

Conversation

victorlin
Copy link
Member

@victorlin victorlin commented Jun 27, 2023

Description of proposed changes

These changes enforce the types that are sprinkled in the codebase here and there, and ensure future types are checked.

Related issue(s)

Testing

What steps should be taken to test the changes you've proposed?
If you added or changed behavior in the codebase, did you update the tests, or do you need help with this?

  • Mypy is run via Pytest in CI
  • Checks fail on first commit
  • Checks pass on final commit

Checklist

  • Add a message in CHANGES.md summarizing the changes in this PR that are end user focused. Keep headers and formatting consistent with the rest of the file.

@victorlin victorlin self-assigned this Jun 27, 2023
@codecov
Copy link

codecov bot commented Jun 27, 2023

Codecov Report

Patch coverage: 66.66% and project coverage change: -0.01 ⚠️

Comparison is base (45f54da) 68.96% compared to head (ebf4c97) 68.96%.

❗ Current head ebf4c97 differs from pull request most recent head a5c173a. Consider uploading reports for the commit a5c173a to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1244      +/-   ##
==========================================
- Coverage   68.96%   68.96%   -0.01%     
==========================================
  Files          64       64              
  Lines        6944     6946       +2     
  Branches     1693     1696       +3     
==========================================
+ Hits         4789     4790       +1     
  Misses       1851     1851              
- Partials      304      305       +1     
Impacted Files Coverage Δ
augur/io/shell_command_runner.py 91.89% <0.00%> (ø)
augur/titer_model.py 70.14% <ø> (ø)
augur/utils.py 72.54% <0.00%> (+0.29%) ⬆️
augur/io/metadata.py 95.15% <50.00%> (-1.15%) ⬇️
augur/filter/include_exclude_rules.py 97.74% <100.00%> (ø)
augur/filter/subsample.py 98.57% <100.00%> (ø)
augur/parse.py 89.41% <100.00%> (ø)
augur/validate.py 74.70% <100.00%> (ø)

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

@victorlin victorlin force-pushed the victorlin/add-type-checking branch 6 times, most recently from 0e7a2b3 to 04e7027 Compare June 29, 2023 18:59
@corneliusroemer corneliusroemer changed the title Add type checking Add type checking (mypy) Jun 29, 2023
@victorlin
Copy link
Member Author

@corneliusroemer you read my mind! I was going to include pyright here but I think it's better to split that out into a separate PR.

@victorlin victorlin marked this pull request as ready for review June 29, 2023 20:34
@victorlin victorlin requested a review from a team June 29, 2023 20:34
@victorlin victorlin mentioned this pull request Jun 29, 2023
4 tasks
Copy link
Contributor

@joverlee521 joverlee521 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exciting to see type checks getting added to Augur! I left some minor comments.

augur/io/metadata.py Outdated Show resolved Hide resolved
augur/parse.py Outdated Show resolved Hide resolved
@victorlin victorlin force-pushed the victorlin/add-type-checking branch from bf4ff71 to 8c19b75 Compare June 30, 2023 00:23
augur/titer_model.py Show resolved Hide resolved
augur/io/metadata.py Outdated Show resolved Hide resolved
The codebase has sprinkles of typing here and there. Previously, they
were not checked in any way and purely for extra context to readers of
the code. Now, they are checked and enforced through mypy.

There are several mypy errors when run against the code as-is. The
following commits will address those errors.

Setup was largely copied from existing mypy usage in Nextstrain CLI,
including the way mypy is run via pytest.
For each external package that was missing types, add a separate type
packages if available. Otherwise, ignore the absence of types.
The violations are surrounded by existing context that justify
exceptions.
Minor non-functional changes to appease the type-checker. To me, the
changes also better describe the code's purpose.
This shows the effects of an existing bug, to be fixed in the following
commit.
@victorlin victorlin force-pushed the victorlin/add-type-checking branch from 8c19b75 to 2a74460 Compare June 30, 2023 20:31
Previously, if --fix-dates was not specified, it would behave as if
--fix-dates='monthfirst' were used.

This change was inadvertently left out of e82d314.
Previous code ran without errors because `fix_dates` is the name of a
function.

The bug was noticed by a Mypy warning hinting that `fix_dates` will
always be truthy.
The type of the delimiter parameter to csv.Sniffer.sniff is str, so
that's what should be accepted here.

Also check to ensure that delimiters are single characters.
@victorlin victorlin force-pushed the victorlin/add-type-checking branch from ebf4c97 to a5c173a Compare June 30, 2023 20:41
@victorlin victorlin merged commit 9e8c9e0 into master Jun 30, 2023
27 checks passed
@victorlin victorlin deleted the victorlin/add-type-checking branch June 30, 2023 20:57
@victorlin victorlin mentioned this pull request Jul 3, 2023
3 tasks
@@ -290,7 +290,7 @@ def available_cpu_cores(fallback: int = 1) -> int:
# naively use all 24, we'd end up with two threads across the 12 cores.
# This would degrade performance rather than improve it!
return len(os.sched_getaffinity(0))
except:
else:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a semantic change here in the edge case, because os.sched_getaffinity(0) can fail and raise a Python exception. When it does, Augur will be broken with the change to if/else instead of falling back to os.cpu_count() with the original try/except.

I think this change should be reverted and mypy appeased another way.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tsibley
Copy link
Member

tsibley commented Jul 5, 2023

Hooray for types! \o/

@victorlin victorlin mentioned this pull request Jul 5, 2023
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

parse: Dates are changed when --fix-dates is not specified
3 participants