Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update typing to 3.12 #5252

Open
dchiquito opened this issue Apr 22, 2024 · 4 comments
Open

Update typing to 3.12 #5252

dchiquito opened this issue Apr 22, 2024 · 4 comments
Labels
C-compat A discrepancy between RustPython and CPython

Comments

@dchiquito
Copy link
Contributor

Feature

The typing module needs to be updated to 3.12. It seems that it was last updated to 3.9, so it's missing several major versions worth of features, like Unpack, generics, and type aliases using the type keyword.

Not having these typing features impedes the process of updating modules that use, which presumably will be most new stuff going forward.

I estimate that updating will require:

  • Moving some abstract base classes from typing.py into a new native module typing.rs, specifically TypeVar, ParamSpec, TypeVarTuple, ParamSpecArgs, ParamSpecKwards, TypeAliasType, and Generic.
  • Adding new syntax like type type aliases and generics to the parser. It seems like this is already included in https://github.com/RustPython/Parser/blob/main/parser/src/python.lalrpop
  • Adding new instructions which generate/consume type information to compiler.rs and frame.rs. For example, def foo[T](x: T): ... must create a new TypeVar(T) and inject it into the correct scope so that it is available as an argument annotation. This requires a new instruction defined in bytecode.rs which is emitted by compiler.rs and consumed by frame.rs.
  • Copying typing.py and test_typing.py from the CPython code base.

Python Documentation or reference to CPython source code

https://docs.python.org/3/library/typing.html

@dchiquito dchiquito added the C-compat A discrepancy between RustPython and CPython label Apr 22, 2024
@dchiquito
Copy link
Contributor Author

I've been chewing on this for a bit, but it's pretty slow going since I'm also learning about a lot of compilation internals. As soon as I have something that is mostly compatible with the 3.12 typing.py I will submit a partial PR to get some incremental feedback.

@youknowone
Copy link
Member

That's awesome. Please feel free to ask anything if you stuck on something.

About typing module, if it helps, mixing Rust and Python implementation is limited but still possible. #5254 is doing it for ExceptionGroup.

@hydrogen602
Copy link
Contributor

hydrogen602 commented May 12, 2024

I don't know if this is already known, but I just wanted to add to the 3.12 typing discussion that if not running in release mode, syntax like def foo[T](): pass causes a panic in the parser at core/src/source_code.rs:139:9

@dchiquito
Copy link
Contributor Author

That problem should be resolved as of #5319 / RustPython/Parser#120. Thanks for reporting!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-compat A discrepancy between RustPython and CPython
Projects
None yet
Development

No branches or pull requests

3 participants