Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add mypy typing #203

Merged
merged 11 commits into from
Dec 19, 2022
Merged

add mypy typing #203

merged 11 commits into from
Dec 19, 2022

Conversation

sbdchd
Copy link
Contributor

@sbdchd sbdchd commented Nov 22, 2022

Adds basic support for #199

Only covers the top level functions

Also removes support for python 3.6 as that doesn't support from __future__ import annotations

@@ -110,7 +111,7 @@ def get_lower_attrib(name):
return re.sub(r".*\.", "", name).lower()


class DublinCoreExtractor(object):
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got rid of some python2 style stuff

@@ -53,7 +56,7 @@ def lxmlDomNodeType(node):
return Node.NOTATION_NODE


class DomHtmlMixin(object):
class DomHtmlMixin:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mixins can be typed in python but it's a little verbose and doesn't seem key for this first pass

I think getting the top level user facing API properly typed is the primary goal of this PR

@sbdchd
Copy link
Contributor Author

sbdchd commented Dec 14, 2022

@BurnzZ when you get a chance

@@ -66,7 +65,6 @@ def get_version():
"Natural Language :: English",
"Operating System :: OS Independent",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.6",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if we should drop 3.6 now. What are the missing annotations from 3.6 that you need? Are they available in https://pypi.org/project/typing-extensions/ ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the issue with python 3.6 is that it doesn't support from __future__ import annotations (which makes annotations lazily evaluated)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lopuhin do you foresee anything breaking in some of our stuff if we drop 3.6? I'm inclined to drop it here since it reached its EOL for almost a year now.

Moreover, I think it'd be great to formally support 3.10 and 3.11 while we're at it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BurnzZ I'm +1 to drop python 3.6 since it's EOL, and support for 3.10 and 3.11 would be great too :)

extruct/_extruct.py Show resolved Hide resolved
@@ -1,3 +1,4 @@
# mypy: disallow_untyped_defs=False
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's enforce typing in the tests as well.

One thing we've learned is that the test themselves test the type annotations of the package itself. For example: https://github.com/scrapinghub/web-poet/pull/78/files#diff-f46006e3f43ffb1dd5d6862005427f6620f4dcfb1fa2f883d8482550069eeecc

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I agree, I think it can be a gradual thing, like first we type the top level, then we gradually add types for the rest of the package

mypy.ini Outdated Show resolved Hide resolved
@sbdchd sbdchd requested a review from BurnzZ December 14, 2022 04:05
@BurnzZ
Copy link
Contributor

BurnzZ commented Dec 16, 2022

@sbdchd I think we can merge this PR after adding a formal support for 3.10 and 3.11, to ensure that the new typing annotations work for them as well.

@sbdchd sbdchd requested review from lopuhin and removed request for BurnzZ December 17, 2022 01:21
Copy link
Contributor

@BurnzZ BurnzZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @sbdchd , this is a great first start for a full annotation support! 🚀

@BurnzZ BurnzZ merged commit 60bc7eb into scrapinghub:master Dec 19, 2022
@sbdchd sbdchd deleted the steve/type-toplevel branch December 20, 2022 03:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants