PEP 612 tracking issue #8645
Comments
|
Your plan looks good to me! I don't expect that #1317 or #5738 will be blockers for this, though they may affect some use cases (which likely don't work currently anyway). In particular, decorating generic functions might not work. I'm planning to work on one or both of those issues in the near future, since they are really quite bad and have been open for a long time. Please don't hesitate to ask further questions here (or email be directly). |
|
I encourage you to start the work on this. Most likely I will just accept the PEP, given that we've seen no further feedback on typing-sig. |
|
Thanks for the kick! I threw some hours at this today, in particular I managed to get through a good amount of the somewhat tedious refactoring caused by generalising TypeVar. I've made a good amount of progress for typeanal of ParamSpec. Branch is still at https://github.com/hauntsaninja/mypy/tree/pep612 |
|
Maybe you can submit it as a PR and mark it Draft? I personally find it hard to follow a change like this as a branch in someone else's repo. |
|
Sure, I opened a draft PR here: #9250 All feedback welcome!
|
Linking #8645 Like #9339, much of this is in #9250. However, this PR is a little less self contained, in that it causes several future TODOs. A lot of the changes here are necessitated by changing the type of CallableType.variables to Sequence[TypeVarLikeDef]. This is nice, because it informs me where I need to make changes in the future / integrates a little bit better with existing type var stuff.
|
So how far have you gotten with this? I see two merged PRs but am afraid to look. One comment: we're currently looking into support for PEP 612 in typing.py for 3.10, and after that in typing_extensions.py for previous Python versions, so it would be nice if these were also supported: from typing_extensions import ParamSpec, Concatenate
from typing import ParamSpec, ConcatenateWe could actually use some feedback on the PR we're working on: the contributor is currently looking into fairly strict checking of the arguments, but I'm wondering if that's useful -- we can't go as far as what you did in mypy_extensions.pyi ( |
|
There hasn't been much progress since my last comment; I sort of deprioritised this in favour of working on other mypy issues that seemed more pressing. But what exists should already support both typing_extensions and typing: Line 3109 in 35e5031 I very much agree that ParamSpec shouldn't validate much at runtime. I think Callable already validates too much, which will necessitate hacks for the backport of ParamSpec to typing_extensions like https://github.com/facebook/pyre-check/blob/0ba810e07635955cbcdef7cd2ab3722ab01dde5b/pyre_extensions/__init__.py#L75 I'll take a look at that PR soon. |
|
I currently get a traceback when using ParamSpec in code: I'd like to make some progress with this -- I don't think I would be very useful with this traceback, but I can do things like adding ParamSpec etc. to typing_extensions (so they can be used with earlier Python versions), both in typeshed and in the actual typing_extensions.py module, and also do a release of the latter (once it's ready). |
|
Huh, there are TODOs all over the code from typing import Callable, ParamSpec, TypeVar
P = ParamSpec("P")
T = TypeVar("T")
def wlog(func: Callable[P, T]) -> Callable[P, T]:
return func
@wlog
def add(a: int, b: int) -> int:
return a+bI'll see if I can do something about it. (UPDATE: I should have read your update a few comments earlier before reporting a traceback. :-) |
|
Fascinating. I'm hitting the very problem described in PEP 612's Motivation chapter. Would be great to have a solution for this. |
|
@sebastian-philipp I think #10883 and #10866 and #10862 does enough to let you use ParamSpec in your projects without breaking downstream mypy users, but you'd need to use pyre or another checker to make sure your own usage is valid |
|
well, for now I'd like to stick with mypy. Switching to a different type checker, especially one that doesn't support getting installed via pip is a non-trivial task. |
Add a as_result() helper to make a decorator to turn a function into one that returns a Result: Regular return values are turned into Ok(return_value). Raised exceptions of the specified exception type(s) are turned into Err(exc). The decorator is signature-preserving, except for wrapping the return type into a Result, of course. For type annotations, this depends on typing.ParamSpec which requires Python 3.10+ (or use typing_extensions); see PEP612 (https://www.python.org/dev/peps/pep-0612/). This is currently not fully supported by Mypy; see python/mypy#8645 Calling decorated functions works without errors from Mypy, but will not be type-safe, i.e. it will behave as if it is calling a function like f(*args: Any, **kwargs: Any) Fixes rustedpy#33.
Add a as_result() helper to make a decorator to turn a function into one that returns a Result: Regular return values are turned into Ok(return_value). Raised exceptions of the specified exception type(s) are turned into Err(exc). The decorator is signature-preserving, except for wrapping the return type into a Result, of course. For type annotations, this depends on typing.ParamSpec which requires Python 3.10+ (or use typing_extensions); see PEP612 (https://www.python.org/dev/peps/pep-0612/). This is currently not fully supported by Mypy; see python/mypy#8645 Calling decorated functions works without errors from Mypy, but will not be type-safe, i.e. it will behave as if it is calling a function like f(*args: Any, **kwargs: Any) Fixes rustedpy#33.
Add a as_result() helper to make a decorator to turn a function into one that returns a Result: Regular return values are turned into Ok(return_value). Raised exceptions of the specified exception type(s) are turned into Err(exc). The decorator is signature-preserving, except for wrapping the return type into a Result, of course. For type annotations, this depends on typing.ParamSpec which requires Python 3.10+ (or use typing_extensions); see PEP612 (https://www.python.org/dev/peps/pep-0612/). This is currently not fully supported by Mypy; see python/mypy#8645 Calling decorated functions works without errors from Mypy, but will not be type-safe, i.e. it will behave as if it is calling a function like f(*args: Any, **kwargs: Any) Fixes rustedpy#33.
|
@NeilGirdhar, the |
|
I think at least mypy should complain where the unusable instance is created in addition to when the unusable instance is used: @sync_to_async # ParamSpec cannot be used with generic functions
def sleep(delay: float, value: T) -> T:
time.sleep(delay)
return value |
Add a as_result() helper to make a decorator to turn a function into one that returns a Result: Regular return values are turned into Ok(return_value). Raised exceptions of the specified exception type(s) are turned into Err(exc). The decorator is signature-preserving, except for wrapping the return type into a Result, of course. For type annotations, this depends on typing.ParamSpec which requires Python 3.10+ (or use typing_extensions); see PEP612 (https://www.python.org/dev/peps/pep-0612/). This is currently not fully supported by Mypy; see python/mypy#8645 Calling decorated functions works without errors from Mypy, but will not be type-safe, i.e. it will behave as if it is calling a function like f(*args: Any, **kwargs: Any) Fixes rustedpy#33.
Add a as_result() helper to make a decorator to turn a function into one that returns a Result: Regular return values are turned into Ok(return_value). Raised exceptions of the specified exception type(s) are turned into Err(exc). The decorator is signature-preserving, except for wrapping the return type into a Result, of course. For type annotations, this depends on typing.ParamSpec which requires Python 3.10+ (or use typing_extensions); see PEP612 (https://www.python.org/dev/peps/pep-0612/). This is currently not fully supported by Mypy; see python/mypy#8645 Calling decorated functions works without errors from Mypy, but will not be type-safe, i.e. it will behave as if it is calling a function like f(*args: Any, **kwargs: Any) Fixes rustedpy#33.
Why is it incorrect behavior?
Yes, that makes sense. It seems like it could work in general though—not just with callables. If MyPy ever supports higher-kinded type variables, this kind of functions of generic will need to be supported, I think. |
Good new is that pylance likes it and copies over the singature and everything. Bad news is that mypy does not support this yet python/mypy#8645 Other minor bad news is that non_generative is not typed. I've tried using a protocol like the one in the comment but the signature is not ported over by pylance, so it's probably best to just live without it to have the correct signature. notes from mike: these three decorators are at the core of getting the library to be typed, more good news is that pylance will do all the things we like re: public_factory, see microsoft/pyright#2758 (comment) . For @_generative, we will likely move to using pep 673 once mypy supports it which may be soon. but overall having the explicit "return self" in the methods, while a little inconvenient, makes the typing more straightforward and locally present in the files rather than being decided at a distance. having "return self" present, or not, both have problems, so maybe we will be able to change it again if things change as far as decorator support. As it is, I feel like we are barely squeaking by with our decorators, the typing is already pretty out there. Change-Id: Ic77e13fc861def76a5925331df85c0aa48d77807 References: #6810
|
I am facing issue with the following code: from typing import Callable, Generic
from typing_extensions import ParamSpec
P = ParamSpec("P")
class Signal(Generic[P]):
def __init__(self) -> None:
pass
def subscribe(self, f: Callable[P, None]) -> None:
pass
def __call__(self, *args: P.args, **kwargs: P.kwargs) -> None:
pass
class M:
def __init__(self) -> None:
self.sig: Signal[[int, str]] = Signal()
def emit(self) -> None:
self.sig(1, 2)Checking it with whereas
P.S.: if which keeps on occurring until removal of mypy's cache entry corresponding to the file containing the |
|
Hi, I face issues, when using from typing_extensions import ParamSpec
from typing import Callable
P = ParamSpec("P")
def foo(func: Callable[P, None]):
def _inner(*args: P.args, **kwargs: P.kwargs):
x = kwargs.pop("x")Error: The code works at runtime. Not sure if it's part of the PEP, that this is possible I opened #12386 for that |
|
We also have
topic-paramspec
|
This PR adds a new Parameters proper type to represent ParamSpec parameters (more about this in the PR), along with supporting the Concatenate operator. Closes #11833 Closes #12276 Closes #12257 Refs #8645 External ref python/typeshed#4827 Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com> Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com>
|
I believe, |
|
#11847 made a lot of progress to full implementation. There are a few more (Someone please correct me if I am wrong) |
|
Yeah, I think users should be able to expect PEP 612 to work, and remaining work is better tracked as individual bugs |
|
@hauntsaninja Thanks a lot for working on this feature! Do you happen to know when the new version of mypy which includes it is going to be released? |
|
(Others did much more than I did here) mypy 0.950 should already support most things and was released about a month ago. #12807 for tracking the next release |
Hello!
I got a little scared off from working on this by the many other enthusiastic people volunteering their time in #8373. However, there doesn't seem to have been much progress; since I have some extra time now, I figured sharing what I've got could kickstart things. Apologies if someone — or whatever a GSoC is — is already working on it :-) Branch is visible at https://github.com/hauntsaninja/mypy/tree/pep612, happy to make it a draft PR if that would help anything.
Overall seems like the plan would be:
Since ParameterSpecification is just a juiced up type variable, it makes sense to try and re-use existing logic for TypeVar as much as possible (especially if later PEPs make use of the bounds and variance arguments that PEP 612 reserves). In my branch, I've chosen to add a base class TypeVarLikeX in a couple different class hierarchies, e.g. TypeVarLikeExpr, TypeVarLikeDef. Since I'm not super familiar with mypy's code, this also helps me stumble upon things that I might need to change (it feels great trusting mypy to help improve mypy!). But this is a fairly large refactor (the one-line change in the details snippet alone will require changes in a dozen files), and could maybe affect like plugins or something, so I thought I'd post this before exploring further.
I'm also a little worried about issues like #1317 and #5738. I don't yet know the code well enough to anticipate what issues we might run into, but I'd welcome any thoughts on interactions between existing TypeVar code and any PEP 612 implementation.
The text was updated successfully, but these errors were encountered: