stubgen: Support TypedDict alternative syntax

[hamdanal]

Fixes #14681

[AlexWaygood]

Mypy actually now supports generic TypedDicts defined with the call syntax, in which case TypedDict wouldn't be the only base (the rewritten version using the class-based syntax would use multiple inheritance with Generic[T] in this playground example): https://mypy-play.net/?mypy=latest&python=3.11&gist=fbeb5bbd0c3036b7327fc65fff0c9a9d

I think it's reasonable not to handle generic TypedDicts defined using the call syntax in this PR, since they're unlikely to come up much. But probably worth a TODO comment?

[hamdanal]

TIL, thank you.
I'll add a TODO for now. Generic TypedDict requires keeping track of TypeVars defined in the file which deserves its own PR IMO.

[AlexWaygood]

What if a user does something like

Foo = TypedDict("Foo", {"a": int}, b=str, d=bytes)

Defining TypedDicts like this using keyword arguments is deprecated at runtime, and has never been supported by mypy — but according to the typing docs it is (or was until recently, when we deprecated it) a supported way of creating a new TypedDict type: https://docs.python.org/3/library/typing.html#typing.TypedDict

[hamdanal]

Right again Alex. Indeed I forgot about the keyword syntax.
The example you gave however is invalid at runtime:

Python 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from typing import TypedDict
>>> Foo = TypedDict("Foo", {"a": int}, b=str, d=bytes)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.10/typing.py", line 2435, in TypedDict
    raise TypeError("TypedDict takes either a dict or keyword arguments,"
TypeError: TypedDict takes either a dict or keyword arguments, but not both

this works:

>>> from typing import TypedDict
>>> Foo = TypedDict("Foo", a=int, b=str, d=bytes)
>>> Foo.__required_keys__
frozenset({'b', 'd', 'a'})
>>> Foo.__optional_keys__
frozenset()

I'll update the PR to add support for the keyword syntax.

[AlexWaygood]

I still think it would be good to be more cautious here. What if somebody runs stubgen on code like this?

T = TypedDict("T", {"a": int}, b=str, total=False)

Now, you'll tell me that this code raises TypeError at runtime, and indeed it does! But that doesn't mean that stubgen should crash (or do something incorrect) if it's run on code like this. We should just emit T: Incomplete and move on to the next thing in the file.

[hamdanal]

I added more checks for invalid cases including this one. I also added a test for importing from typing_extensions.

With all the checks for invalid uses, the TypedDict handling part looks much more conservative than other parts of stubgen. I am not sure what to make of this, it is not necessarily a positive or negative thing, just something I noticed.

[AlexWaygood]

With all the checks for invalid uses, the TypedDict handling part looks much more conservative than other parts of stubgen. I am not sure what to make of this, it is not necessarily a positive or negative thing, just something I noticed.

Hmm, well, I'm not really an expert on stubgen, so I can't really comment on decisions that have previously been taken. But in my opinion, it's always good for static analysis tools to be as paranoid as possible about the kind of thing that might be fed to them :)

[AlexWaygood]

Starting to look pretty good, from what I can tell! Could you add a test case where TypedDict is imported from typing_extensions in the source code stubgen's being run on?

[AlexWaygood]

Not an expert on stubgen, but this LGTM as far as I can tell!