Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add [mypy].extra_type_stubs #13180

Merged
merged 2 commits into from Oct 11, 2021

Conversation

Eric-Arellano
Copy link
Contributor

Follows up on #12597. While that change was necessary as explained there, we've had feedback from two users that they want to retain the ability to specify type stubs via option rather than normal runtime dependencies. For example, one user wants to avoid an increase in .pex size (because one of the stubs incorrectly depends on MyPy).

Ideally, we should probably re-envision third-party type stubs wholesale so that our code distinguishes between normal dependencies vs. type stubs dependencies. When running MyPy, we use type stubs. When packaging, for example, we know to leave them off. But that requires design work and time to implement.

In the meantime, we can add [mypy].extra_type_stubs, which satisfies the different requirements:

There is one weird edge: these type stubs don't have a lockfile associated with them. Those can't use the tool lockfile because we cannot tell MyPy to point into its tool lockfile (#12597). It would require a lot of new infrastructure to have a dedicated lockfile for extra_type_stubs. So, instead, if you do care about lockfiles, you should use type stubs like normal dependencies rather than using this option.

[ci skip-rust]
[ci skip-build-wheels]

# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
@Eric-Arellano
Copy link
Contributor Author

There is one weird edge: these type stubs don't have a lockfile associated with them.

Not sure if I should put this into the help message? It's a little weird to do because Pants 2.8 lockfile support isn't really there to begin with.

@Eric-Arellano Eric-Arellano added this to the 2.7.x milestone Oct 8, 2021
Copy link

@phlax phlax left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Eric-Arellano thanks for resolving this

visually the code looks good, but i was unable to do an editable install so wasnt able to test

thanks again

@benjyw
Copy link
Sponsor Contributor

benjyw commented Oct 9, 2021

Ideally, we should probably re-envision third-party type stubs wholesale so that our code distinguishes between normal dependencies vs. type stubs dependencies. When running MyPy, we use type stubs. When packaging, for example, we know to leave them off. But that requires design work and time to implement.

Presumably this relates to #12794 ? Type stubs would be a "compile"-time dependency?

Copy link
Sponsor Member

@stuhood stuhood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As @benjyw mentioned, we should fix this via #12794. But this is useful in the meantime.

[ci skip-rust]
[ci skip-build-wheels]

# Conflicts:
#	src/python/pants/backend/python/typecheck/mypy/rules_integration_test.py
@Eric-Arellano Eric-Arellano enabled auto-merge (squash) October 11, 2021 17:03
@Eric-Arellano Eric-Arellano merged commit 6219e67 into pantsbuild:main Oct 11, 2021
@Eric-Arellano Eric-Arellano deleted the mypy-type-stubs branch October 11, 2021 17:43
Eric-Arellano added a commit to Eric-Arellano/pants that referenced this pull request Oct 11, 2021
Follows up on pantsbuild#12597. While that change was necessary as explained there, we've had feedback from two users that they want to retain the ability to specify type stubs via option rather than normal runtime dependencies. For example, one user wants to avoid an increase in `.pex` size (because one of the stubs incorrectly depends on MyPy).

Ideally, we should probably re-envision third-party type stubs wholesale so that our code distinguishes between normal dependencies vs. type stubs dependencies. When running MyPy, we use type stubs. When packaging, for example, we know to leave them off. But that requires design work and time to implement.

In the meantime, we can add `[mypy].extra_type_stubs`, which satisfies the different requirements:

* We don't install these into the "tool PEX", which is necessary per pantsbuild#12597.
* Type stubs continue to work when done as a normal requirement. These are "extra".

There is one weird edge: these type stubs don't have a lockfile associated with them. Those can't use the tool lockfile because we cannot tell MyPy to point into its tool lockfile (pantsbuild#12597). It would require a lot of new infrastructure to have a dedicated lockfile for `extra_type_stubs`. So, instead, if you do care about lockfiles, you should use type stubs like normal dependencies rather than using this option.

[ci skip-rust]
[ci skip-build-wheels]
Eric-Arellano added a commit that referenced this pull request Oct 11, 2021
Follows up on #12597. While that change was necessary as explained there, we've had feedback from two users that they want to retain the ability to specify type stubs via option rather than normal runtime dependencies. For example, one user wants to avoid an increase in `.pex` size (because one of the stubs incorrectly depends on MyPy).

Ideally, we should probably re-envision third-party type stubs wholesale so that our code distinguishes between normal dependencies vs. type stubs dependencies. When running MyPy, we use type stubs. When packaging, for example, we know to leave them off. But that requires design work and time to implement.

In the meantime, we can add `[mypy].extra_type_stubs`, which satisfies the different requirements:

* We don't install these into the "tool PEX", which is necessary per #12597.
* Type stubs continue to work when done as a normal requirement. These are "extra".

There is one weird edge: these type stubs don't have a lockfile associated with them. Those can't use the tool lockfile because we cannot tell MyPy to point into its tool lockfile (#12597). It would require a lot of new infrastructure to have a dedicated lockfile for `extra_type_stubs`. So, instead, if you do care about lockfiles, you should use type stubs like normal dependencies rather than using this option.

[ci skip-rust]
[ci skip-build-wheels]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants