Ensure types are consistent and turn on `mypy` #1162

sarahmonod · 2023-10-02T17:47:43Z

This is a draft PR to start the discussion on adding type information to github3.py.

The first commit was necessary to get the tests to pass for me, I didn't investigate any further than just mark the failing tests as xfail as they were failing for the default branch.

The second commit is the minimal set of change I could think of that would make running mypy return a success.

The third commit is just a few of the types that could be added to github3.py. I'm happy to add more, but would like to know what the project maintainers think of this before I do.

It looks like `betamax` is failing to appropriately mock the `close` method in its mock connection. Signed-off-by: Gus Monod <gmonod1@bloomberg.net>

Signed-off-by: Gus Monod <gmonod1@bloomberg.net>

sigmavirus24

Much of this looks like it should satisfy mypy but it doesn't actually provide useful types to users. GitHubCore isn't something that people should be caring about. They should be getting return types that point to the object they care about and the methods and attributes on it. As it stands, if people use this, it will lead to them needing tonnes of type: ignore whenever they access a method or attribute for the class in particular.

sigmavirus24 · 2023-10-03T14:24:31Z

src/github3/checks.py

@@ -45,7 +45,7 @@ def _update_attributes(self, pull):
    def _repr(self):
        return f"<CheckPullRequest [#{self.number}]>"

-    def to_pull(self):
+    def to_pull(self, conditional: bool = False):


This should return -> t.Optional[pulls.PullRequest]

sigmavirus24 · 2023-10-03T14:25:04Z

src/github3/events.py

-    def to_review_comment(self):
+    def to_review_comment(
+        self, conditional: bool = False
+    ) -> models.GitHubCore:


This is too generic. We can be more specific here.

sigmavirus24 · 2023-10-03T14:25:50Z

src/github3/git.py

@@ -382,6 +346,42 @@ def recurse(self):
        return self._instance_or_null(Tree, json)


+class CommitTree(models.GitHubCore):


Why are you moving this definition? That seems unnecessary

This is because of a bug in mypy, let me find the reference.

sigmavirus24 · 2023-10-03T14:27:51Z

src/github3/models.py

    @_api.setter
    def _api(self, uri):
        if uri:
            self._uri = self._uri_parse(uri)
        self.url = uri

+    @staticmethod
+    def _uri_parse(uri):


Again, there's no point or value apparent in moving this

This is because mypy cannot deal with @property getters and setters not being right next to each other: python/mypy#1465

Well that's truly infuriating

sigmavirus24 · 2023-10-03T14:28:56Z

src/github3/models.py

    def _iter(
        self,
        count: int,
        url: str,
-        cls: t.Type[T],
-        params: t.Optional[t.Mapping[str, t.Optional[str]]] = None,
+        cls: t.Type["GitHubCore"],


Technically this is true, but it's not accurate. _iter returns a GitHubIterator that returns a given type, which we can tie to cls.

Oh good point, let me see if I can fix that.

sigmavirus24 · 2023-10-03T14:29:34Z

src/github3/orgs.py

@@ -491,7 +491,7 @@ def add_repository(self, repository, team_id):  # FIXME(jlk): add perms
    @requires_auth
    def blocked_users(
        self, number: int = -1, etag: t.Optional[str] = None
-    ) -> t.Generator[users.ShortUser, None, None]:
+    ) -> t.Iterator[users.ShortUser]:


I believe t.Iterator is likely going to cause issues as GitHubIterator has publicly documented additional methods which this type does/will not reveal

Do you mean you expect people to be using methods from t.Generator or GitHubIterator?

I mean the latter. The old annotation was certainly also wrkng

sigmavirus24 · 2023-10-03T14:30:29Z

src/github3/orgs.py

@@ -749,7 +749,7 @@ def create_team(
                getattr(r, "full_name", r) for r in (repo_names or [])
            ],
            "maintainers": [
-                getattr(m, "login", m) for m in (maintainers or [])
+                str(getattr(m, "login", m)) for m in (maintainers or [])


Why is this necessary but getattr(r, "full_name", r) isn't?

sarahmonod · 2023-10-03T16:02:10Z

Much of this looks like it should satisfy mypy but it doesn't actually provide useful types to users. GitHubCore isn't something that people should be caring about. They should be getting return types that point to the object they care about and the methods and attributes on it. As it stands, if people use this, it will lead to them needing tonnes of type: ignore whenever they access a method or attribute for the class in particular.

Exactly. I would like to break this work into two pieces, which may or may not be in the same PR. One would make mypy pass, and the other would actually add the types. I don't think that adding types is possible/a good idea until mypy passes, and that is why I'm trying to gather feedback on the best way to do that is, before I move on to adding a lot of types.

sigmavirus24 · 2023-10-11T13:28:37Z

I see your point about wanting to break this up into smaller pieces of work. I'll approve and merge this since mypy does seem to not be unhappy with it. I would want to progressively make mypy more strict though as we enforce things

sarahmonod force-pushed the types branch 5 times, most recently from c811376 to bd4e1b0 Compare October 2, 2023 18:32

tests: Mark failing tests as xfail

9d6124c

It looks like `betamax` is failing to appropriately mock the `close` method in its mock connection. Signed-off-by: Gus Monod <gmonod1@bloomberg.net>

sarahmonod force-pushed the types branch from 88c1bbe to bc83ae7 Compare October 2, 2023 18:35

sarahmonod marked this pull request as ready for review October 3, 2023 12:55

sarahmonod force-pushed the types branch from bc83ae7 to d375a35 Compare October 3, 2023 12:58

Ensure files with type hints are consistent

8c37348

Signed-off-by: Gus Monod <gmonod1@bloomberg.net>

sarahmonod force-pushed the types branch from d375a35 to 8c37348 Compare October 3, 2023 13:54

sarahmonod changed the title ~~Add type information~~ Ensure types are consistent and turn on mypy Oct 3, 2023

sigmavirus24 requested changes Oct 3, 2023

View reviewed changes

sigmavirus24 approved these changes Oct 11, 2023

View reviewed changes

sigmavirus24 merged commit 8288151 into sigmavirus24:main Oct 11, 2023
11 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Ensure types are consistent and turn on `mypy` #1162

Ensure types are consistent and turn on `mypy` #1162

sarahmonod commented Oct 2, 2023 •

edited

Loading

sigmavirus24 left a comment

sigmavirus24 Oct 3, 2023

sigmavirus24 Oct 3, 2023

sigmavirus24 Oct 3, 2023

sarahmonod Oct 3, 2023

sigmavirus24 Oct 3, 2023

sarahmonod Oct 10, 2023 •

edited

Loading

sigmavirus24 Oct 11, 2023

sigmavirus24 Oct 3, 2023

sarahmonod Oct 3, 2023

sigmavirus24 Oct 3, 2023

sarahmonod Oct 3, 2023

sigmavirus24 Oct 4, 2023

sigmavirus24 Oct 3, 2023

sarahmonod commented Oct 3, 2023 •

edited

Loading

sigmavirus24 commented Oct 11, 2023

		@@ -382,6 +346,42 @@ def recurse(self):
		return self._instance_or_null(Tree, json)


		class CommitTree(models.GitHubCore):

Ensure types are consistent and turn on mypy #1162

Ensure types are consistent and turn on mypy #1162

Conversation

sarahmonod commented Oct 2, 2023 • edited Loading

sigmavirus24 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sarahmonod Oct 10, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sarahmonod commented Oct 3, 2023 • edited Loading

sigmavirus24 commented Oct 11, 2023

Ensure types are consistent and turn on `mypy` #1162

Ensure types are consistent and turn on `mypy` #1162

sarahmonod commented Oct 2, 2023 •

edited

Loading

sarahmonod Oct 10, 2023 •

edited

Loading

sarahmonod commented Oct 3, 2023 •

edited

Loading