pkg_resources: Expose the type of imports and re-exports

[Avasam]

Summary of changes

• Removed unused resource_dir and resource_dir pkg_resources: Remove unused and shadowed variables #4260
• Removed overriden run_main and run_script pkg_resources: Remove unused and shadowed variables #4260
• Moved declarations of @_call_aside defined globals closer to where they're defined. And after ResourceManager & WorkingSet to avoid forward reference issues pkg_resources type the declared global variables #4267
• Typed the 3 globals set by _declare_state as not None Type pkg_resources._declare_state and make it work statically #4258
• Statically exposed the dynamic imports using a TYPE_CHECKING block and static imports

Works towards #2345 and a big step towards merging typeshed's pkg_resources stubs into setuptools

Pull Request Checklist

• Changes have tests (this is to help with type-checking checks)
• News fragment added in newsfragments/.
  (See documentation for details)

[abravalheri]

This is a bit tricky...

Based on the comments:

• Fix all mypy issues #3979 (comment)
• Fix all mypy issues #3979 (comment)
• Fix all mypy issues #3979 (comment)

I suppose this would be a controversial change (I am not sure if it matters to have the if TYPE_CHECKING in pkg_resources/__init__.py or pkg_resouces/extern/__init__.py, probably the comments apply to both).

At the end of the day, mypy cannot handle MetaPathFinders... I suppose that the ultimate solution would be to use a mypy plugin to "convince" mypy to use the files in the */_vendor folder for the *.extern modules, but that sounds very complicated¹. So there is a trade-off here: complexity of implementing a mypy plugin vs. in the duplication in *.extern modules vs. leave the things unchecked (and loosing some of the benefits of the type analysis).

@jaraco, do you have any opinion about this one?

Footnotes

1. I cannot find a hook for finding modules in the docs: https://mypy.readthedocs.io/en/stable/extending_mypy.html#extending-mypy-using-plugins. So I opened https://github.com/python/mypy/issues/16988 for guidance on this. ↩

[Avasam]

It's slightly different from what I originally did in #3979 in that it's only duplicating what's needed/used. And doing so right next to the equivalent dynamic imports. So imo there's less of a maintenance concern.

---

packaging.requirements is especially important to avoid looking like we're subclassing Any for RequirementParseError and Requirement

With static imports:

W/o the import behind a TYPE_CHECKING:

---

FWIW, I'd rather loose on type safety internally, publicly loose on implicit return types (until all public API return types are explicit), and loose on some super class typing, than going for a mypy plugin. Which also only works on mypy, not pyright, VSCode+Pylance, or PyCharm.

I'll probably end up keeping those changes in my local working branches since it helps me validate the types and understand intended behaviour. Even if it doesn't end up in main.

[abravalheri]

Thank you very much for working on this.

This is probably a new occurrence of the same discussion in #3979 (comment).

In that comment, Jason expressed concerns for redundant imports and how this approach makes it cumbersome to code (my interpretation for why that is not desirable: you have to go around modifying multiple portions of multiple files and if you forget to do it in one portion, something is going to fail - kind of shotgun surgery anti-pattern).

I do acknowledge that for type checking, there seems to be no better solution than the double import, but I think this issue is not something we have reached a final decision yet.

[Avasam]

There's definitely some pros and cons to any approach (including "leaving it as-is") to this situation.
Of course I have my bias/preference to being able to type the package on par with (or better than) typeshed stubs, but agree we can cross that bridge when there's nothing else left. Leaves more time to brew some ideas and see what it affects.

[Avasam]

Setting as draft to make it clear this isn't a priority to review. Will probably end up being the last pkg_resources PR for static typing.