Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add new typecheck goal for MyPy #10212

Merged
merged 6 commits into from Jul 1, 2020

Conversation

Eric-Arellano
Copy link
Contributor

While it would be ideal to have lint run MyPy too, we realized that it does not work well with the --changed-since flag.

Normally, we recommend running ./pants --changed-since=main lint, which only runs over directly changed targets. With MyPy, you will also want to specify --changed-include-dependees=transitive. So, you must either choose between a) running over too few targets if you leave off dependees, or b) wasting time running Black/Flake8/Isort over unnecessary targets.

We want to optimize for the --changed-since use case, especially with local runs when iterating, so this is an important blocker.

[ci skip-rust-tests]

# Rust tests will be skipped. Delete if not intended.
[ci skip-rust-tests]
However, some linters may need to partition their batch of LinterFieldSets and thus may need to
return multiple results. For example, many Python linters will need to group by interpreter
compatibility.
However, some linters may need to partition their input and thus may need to return multiple
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only renames in this file.

Comment on lines -229 to -231
@pytest.mark.skip(
reason="Figure out how to get MyPy working with both namespace packages and source roots. See https://github.com/Eric-Arellano/mypy-debug."
)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See https://pantsbuild.slack.com/archives/CASMF8SJ1/p1593473127209200. I can't figure out how to get this working, even without Pants. So, I don't want dead code hanging around.

I may take a third look tomorrow..

Comment on lines +49 to +55
class TypecheckResults(Collection[TypecheckResult]):
"""Zero or more TypecheckResult objects for a single type checker.

Typically, type checkers will return one result. If they no-oped, they will return zero results.
However, some type checkers may need to partition their input and thus may need to return
multiple results.
"""
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no need yet to batch things. We expect MyPy to return one single result. But, it's very very little extra code for us to have type checkers return a collection, and I think we'll appreciate the flexibility in the future.

This is more consistent, e.g. making docs easier to approach. Also, there are multiple Python type checkers, so this accomadates any possible future Python type checkers we may add.

# Rust tests will be skipped. Delete if not intended.
[ci skip-rust-tests]
# Rust tests will be skipped. Delete if not intended.
[ci skip-rust-tests]
Copy link
Sponsor Member

@stuhood stuhood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making this change. Just a naming nit.

Comment on lines +66 to +75
class TypecheckOptions(GoalSubsystem):
"""Run type checkers."""

name = "typecheck"

required_union_implementations = (TypecheckRequest,)


class Typecheck(Goal):
subsystem_cls = TypecheckOptions
Copy link
Sponsor Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd recommend shortening this to just "check" for a few reasons:

  1. it aligns with Rust's cargo check, which does similar things: the minimal amount of compilation required to check that code "would" compile
  2. it allows for other similar semantic checks which are not exactly "type checking", but which cost a similar amount (maybe because they require transitive dependencies): for example: cargo check also runs clippy, which is a semantic linter that requires type information.
  3. it's shorter.

@Eric-Arellano
Copy link
Contributor Author

I talked to Benjy about this PR on VC.

We discussed first whether MyPy should be in typecheck or lint. While it would be nice to be in lint, for now, we're thinking typecheck because it's out of scope to do the redesign of --changed-since necessary to get this working (see problem statement). Also, semantically, MyPy is spiritually like compile, only that it's optional.

I think it's possible that we later change MyPy to live in lint after we improve --changed-since. It will still be possible to make that change, just like we originally migrated MyPy from mypy to lint.

--

The next discussion was typecheck vs. check. I acknowledge Rust's use of check, but feel strongly that we should stick to typecheck. Imo, check and lint are synonymous. If you asked an average Python user, I don't think they'd be able to tell you the difference.

When we add Rust, it will be unclear where cargo check should then live, e.g. cargo-check, lint, or typecheck. I argue that it's premature for us to answer this. For example, it's very plausible that we make the --changed-since improvements and merge everything back into lint, so we want to put cargo check in lint. We can adjust as we go.

@Eric-Arellano
Copy link
Contributor Author

Hm. One idea to keep MyPy in lint and still solve the --changed-since issue: add a new option like --mypy-run-on-dependees. If true, mypy/rules.py will get the dependees of the previously specified targets. It's agnostic to how those original targets were populated, it only cares about finding the dependees.

This would allow a user to permanently set --mypy-run-on-dependees in a script and still do the right thing when running ./pants --changed-since=HEAD lint.

What do you both think? A weird point is that it duplicates --changed-dependees=transitive. Fwit, we would possibly want to add --test-run-on-dependees (I don't think --pytest-run-on-dependees). Both these options would default to False to be less surprising.

@stuhood
Copy link
Sponsor Member

stuhood commented Jul 1, 2020

What do you both think? A weird point is that it duplicates --changed-dependees=transitive. Fwit, we would possibly want to add --test-run-on-dependees (I don't think --pytest-run-on-dependees). Both these options would default to False to be less surprising.

That would be very powerful, yea. But I think that if we did that, we'd want to do it via a scoped Subsystem that affected the whole goal... then you could do something like ./pants lint check --${sub}-dependees test --${sub}-dependees to affect the relevant goals.

@benjyw
Copy link
Sponsor Contributor

benjyw commented Jul 1, 2020

I think it's fine not to keep mypy in lint. It is a different type of thing than a linter - half way between a linter and a compiler, and as we've seen, that means it has different characteristics around dependency handling. And we don't even know if users would prefer one over the other even if they were equally easy to do. So rather than add complexity, let's just do this.

Re the name, I tend to agree that typecheck is preferable, despite being longer, since check is not very evocative, and it's unclear what it means compared to lint (or compared to, say, other custom "checks", such as dependency validation, that repos might add).

@Eric-Arellano
Copy link
Contributor Author

Okay, thank you both.

@Eric-Arellano Eric-Arellano merged commit 238b3b7 into pantsbuild:master Jul 1, 2020
@Eric-Arellano Eric-Arellano deleted the typecheck branch July 1, 2020 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants