Skip to content

Conversation

samestep
Copy link
Contributor

@samestep samestep commented Feb 5, 2021

This PR adds a local mypy plugin that warns if you accidentally run mypy using a version that doesn't match the version we install for CI, since this trips people up sometimes when mypy gives errors in some versions (see #51513) but not others.

Test plan:

To check that this doesn't break our mypy test(s) when you have the correct version installed:

python test/test_type_hints.py

To check that this does indeed warn when you have an incorrect mypy version installed, switch to a different version (e.g. 0.782), and run the above command or either of these:

mypy
mypy --config-file=mypy-strict.ini

You should get the following message on stderr:

You are using mypy version 0.782, which is not supported
in the PyTorch repo. Please switch to mypy version 0.770.

For example, if you installed mypy via pip, run this:

    pip install mypy==0.770

Or if you installed mypy via conda, run this:

    conda install -c conda-forge mypy=0.770

@samestep samestep requested review from a team and zdevito February 5, 2021 19:20
@facebook-github-bot
Copy link
Contributor

facebook-github-bot commented Feb 5, 2021

💊 CI failures summary and remediations

As of commit 39f02aa (more details on the Dr. CI page):


  • 1/1 failures introduced in this PR

1 failure not recognized by patterns:

Job Step Action
CircleCI pytorch_linux_bionic_py3_8_gcc9_coverage_test1 Run tests 🔁 rerun

This comment was automatically generated by Dr. CI (expand for details).Follow this link to opt-out of these comments for your Pull Requests.

Please report bugs/suggestions to the (internal) Dr. CI Users group.

Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@samestep has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@janeyx99
Copy link
Contributor

janeyx99 commented Feb 8, 2021

Could this be too strict? Is it the case that most mypy versions work with the one we use, or is it more like using any other version would cause conflicting warnings?

If the first, I have high level questions:

  1. Is it possible to only print this warning when there are mypy failures?
  2. Is it better to fail the mypy job/test versus still have it pass but print the warning in the case that no real mypy errors were emitted?

@samestep
Copy link
Contributor Author

samestep commented Feb 8, 2021

@janeyx99 Good questions. I'm not sure about "most" mypy versions, but for instance we know that mypy 0.800 gives warnings on our current codebase (#51513) where mypy 0.770 does not.

With regard to your two questions, could you clarify more what the motivation would be for printing this warning in fewer cases, or for not failing the local test run? Note that with this PR as-is, the warning will only appear if all of these conditions are met:

  • the developer has mypy installed
  • the developer runs test/test_type_hints.py (or mypy some other way) locally
  • the developer's installed version of mypy doesn't match CI

Given that the error message shows what command to run to correct the mypy version, is there a reason an error would be undesirable in this case?

@janeyx99
Copy link
Contributor

janeyx99 commented Feb 8, 2021

Yes, I was thinking of if the local mypy version is okay (i.e., does not give warnings we don't care about), it might be annoying to see this warning message every time mypy was run. Basically, if a programmer is doing fine with a different mypy version, I'm not sure we want to insist a switch.

@samestep
Copy link
Contributor Author

samestep commented Feb 8, 2021

Yes, I was thinking of if the local mypy version is okay (i.e., does not give warnings we don't care about), it might be annoying to see this warning message every time mypy was run. Basically, if a programmer is doing fine with a different mypy version, I'm not sure we want to insist a switch.

Are we assuming that the developer already has a Python environment (either conda or virtualenv etc) for doing PyTorch development? Because if so then I'm not sure I see the reasoning (since you'd only be changing your mypy version specifically for doing PyTorch development); if not then maybe.

I suppose I could modify the message to say that if you don't already have a Python environment for PyTorch development and are just using the global environment, then you should make one.

@janeyx99
Copy link
Contributor

janeyx99 commented Feb 8, 2021

Yea that's fair. This would be the way to enforce that PyTorch devs should use the same version of mypy used in CI. The enforcement can vary in degree where it's more a suggestion/warning vs a rule, but I'm not sure how much we want to be lax about it so you can decide :)

Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@samestep has imported this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

Copy link
Contributor

@janeyx99 janeyx99 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

coolio 😎

@facebook-github-bot
Copy link
Contributor

@samestep merged this pull request in dac730a.

xsacha pushed a commit to xsacha/pytorch that referenced this pull request Mar 31, 2021
Summary:
This PR adds a local [`mypy` plugin](https://mypy.readthedocs.io/en/stable/extending_mypy.html#extending-mypy-using-plugins) that warns if you accidentally run `mypy` using a version that doesn't match [the version we install for CI](https://github.com/pytorch/pytorch/blob/6045663f391e9bebac31e006a98c1dd381792936/.circleci/docker/common/install_conda.sh#L117), since this trips people up sometimes when `mypy` gives errors in some versions (see pytorch#51513) but not others.

Pull Request resolved: pytorch#51799

Test Plan:
To check that this doesn't break our `mypy` test(s) when you have the correct version installed:
```
python test/test_type_hints.py
```
To check that this does indeed warn when you have an incorrect `mypy` version installed, switch to a different version (e.g. 0.782), and run the above command or either of these:
```
mypy
mypy --config-file=mypy-strict.ini
```
You should get the following message on stderr:
```
You are using mypy version 0.782, which is not supported
in the PyTorch repo. Please switch to mypy version 0.770.

For example, if you installed mypy via pip, run this:

    pip install mypy==0.770

Or if you installed mypy via conda, run this:

    conda install -c conda-forge mypy=0.770
```

Reviewed By: janeyx99

Differential Revision: D26282010

Pulled By: samestep

fbshipit-source-id: 7b423020d0529700dea8972b27afa2d7068e1b12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants