Support for `declared_type`, to give types to decorated functions #3291

sixolet · 2017-05-01T04:53:51Z

This is an implementation of python/peps#242 in mypy_extensions instead of typing for now (but with forward compat when it gets to typing)

JukkaL

Thanks for the PR! Looks good, I just have some ideas about test cases.

JukkaL · 2017-05-02T16:13:15Z

test-data/unit/lib-stub/mypy_extensions.pyi


 T = TypeVar('T')


 def TypedDict(typename: str, fields: Dict[str, Type[T]]) -> Type[dict]: pass

 class NoReturn: pass
+
+def decorated_type(t: Type) -> Callable[[T], T]: pass


The argument type should be Any, since some valid types aren't type objects (e.g. None).

JukkaL · 2017-05-02T16:13:34Z

test-data/unit/check-functions.test

+
+@decorated_type  # E: "decorated_type" must have a type as an argument
+@dec
+def f(): pass


Add reveal_type(f).

JukkaL · 2017-05-02T16:13:44Z

test-data/unit/check-functions.test

+
+@decorated_type()  # E: "decorated_type" takes exactly one argument
+@dec
+def f(): pass


Add reveal_type(f).

JukkaL · 2017-05-02T16:13:56Z

test-data/unit/check-functions.test

+
+@dec
+@decorated_type(Callable[[int], str])  # E: "decorated_type" must be the topmost decorator
+def f(): pass


Add reveal_type(f).

JukkaL · 2017-05-02T16:15:00Z

test-data/unit/check-functions.test

+
+def dec(f: T) -> T: pass
+
+@decorated_type(Callable[[int], str])  # E: Incompatible types (inferred decorated type Callable[[str], str], declared decorated type Callable[[int], str])


Add test where the declared type is a proper supertype of the inferred type. It should be okay -- i.e. test that the subtype checks goes the right way.

JukkaL · 2017-05-02T16:16:09Z

test-data/unit/check-functions.test

+def f(): pass
+
+[builtins fixtures/dict.pyi]
+


Add test cases that use staticmethod and classmethod together with decorated_type.

JukkaL · 2017-05-02T16:18:42Z

Can you also fix the merge conflict?

This needs https://github.com/python/typeshed/pull/1230/files before this is usable outside tests. I'll merge this PR first.

sixolet · 2017-06-15T04:47:16Z

@JukkaL ping seems ready after relevant typeshed change.

gvanrossum · 2017-06-20T21:54:00Z

Not a review, just wondering if this might be a candidate for implementing through the new plugin architecture, once #3534 lands?

gvanrossum · 2017-09-19T21:59:14Z

If we're proposing this as a general typing feature (not something mypy-specific), which I think we are, then this should ideally be placed in typing_extensions, which is now a thing: https://github.com/python/typing/tree/master/typing_extensions.

(I withdraw my suggestion to consider this for the plugin architecture.)

JukkaL

Looks good, just some minor nits. Thanks for implementing this! Apologies about the long review delay -- I got distracted by various things including vacation and internal team changes, and I took a long time to finish the review I started a while ago.

JukkaL · 2017-06-21T15:24:56Z

test-data/unit/check-functions.test

@@ -734,6 +734,188 @@ a = None # type: A
 a.f()
 a.f(None) # E: Too many arguments for "f" of "A"

+[case testMethodWithDeclaredDecoratedType]
+


Style nit: Redundant empty line (and similar cases in other test cases).

JukkaL · 2017-06-21T15:28:08Z

test-data/unit/check-functions.test

+y: int = foo.f(1)  # E: Incompatible types in assignment (expression has type "str", variable has type "int")
+
+reveal_type(foo.f)  # E: Revealed type is 'def (builtins.int) -> builtins.str'
+


Another context where we don't tend to have empty lines.

JukkaL · 2017-06-21T15:29:13Z

test-data/unit/check-functions.test

+from mypy_extensions import declared_type
+
+@declared_type(Callable[[int], str])
+def f(x): pass


Maybe reveal type of x inside f to ensure that the declared type doesn't get propagated.

JukkaL · 2017-09-20T16:09:31Z

extensions/mypy_extensions.py

+    """Declare the type of a declaration.
+
+    This is useful for declaring a more specific type for a decorated class or
+    function definition than the decorator provides as a return value.


I wonder if there is a context where it's useful to use this to decorate a class?

JukkaL · 2017-09-20T16:10:55Z

mypy/semanal.py

@@ -2633,6 +2633,19 @@ def visit_decorator(self, dec: Decorator) -> None:
            elif refers_to_fullname(d, 'typing.no_type_check'):
                dec.var.type = AnyType()
                no_type_check = True
+            elif isinstance(d, CallExpr) and (
+                    refers_to_fullname(d.callee, 'typing.declared_type')


As discussed above, maybe also accept typing_extensions.declared_type.

JukkaL · 2017-09-20T16:11:10Z

mypy/semanal.py

+                    self.fail('"declared_type" takes exactly one argument', d)
+                else:
+                    dec.var.type = self.expr_to_analyzed_type(d.args[0])
+            elif (refers_to_fullname(d, 'typing.declared_type') or


Same as above.

JukkaL · 2017-09-20T16:14:01Z

test-data/unit/check-functions.test

+def dec(f: T) -> T: pass
+
+@declared_type(Callable[[int], str])  # E: Incompatible types (inferred decorated type Callable[[str], str], declared decorated type Callable[[int], str])
+@dec


Also test declared_type together with another decorator that modifies the signature of the function. Test both a good usage and an invalid usage (where the declared type is incompatible).

JukkaL · 2017-09-20T16:15:11Z

test-data/unit/check-functions.test

+
+# Note that the decorated type must account for the `cls` argument  -- It's applied pre-binding
+class Foo:
+    @declared_type(Callable[[Any, int], str])


Test similar case where you use Type[...] type for the cls argument.

JukkaL · 2017-09-20T16:15:34Z

test-data/unit/check-functions.test

+
+# Note that the decorated type must account for the `self` argument  -- It's applied pre-binding
+class Foo:
+    @declared_type(Callable[[Any, int], str])


Test similar case where you use a non-Any type for the self argument, such as Foo here.

ilevkivskyi · 2017-09-20T16:22:48Z

mypy/checker.py

-        e.var.type = sig
+        if e.var.type is not None:
+            # We have a declared type, check it.
+            self.check_subtype(sig, e.var.type, e,


@JukkaL Shouldn't this be other way around?
I thought that the declared type is expected to be more narrow/precise than the inferred.

I think that this is right -- this is not a cast but a declaration. But it would be good to have a test case where the ordering of this check is important.

The problem with this is that it will be then impossible to give a more precise type for a decorated function when it is known. From the discussion on python/peps tracker I somehow understood that this should work like a cast. I however could imagine situations where one might want to declare narrower argument types, so maybe this check is not needed and we can allow both ways?

This can be used to declare a more precise type with respect to Any types (Any is considered a subtype of everything).

OK, I see, I was thinking about hijacking this decorator for #2087

JukkaL · 2018-04-04T15:31:16Z

I'm closing the PR since there hasn't been activity in a long time. Feel free to reopen if you wish to continue working on this.

sixolet added 4 commits April 30, 2017 15:15

First draft of decorated_type. Needs more tests

4e81716

more tests

f15779a

Make first-stage generator inference give up on async coroutines

6af0979

lint

c739cef

sixolet mentioned this pull request May 1, 2017

[WIP] Typeshed support for mypy_extensions.declared_type python/typeshed#1230

Closed

sixolet added 3 commits April 30, 2017 21:57

typo

f0f30f4

update typeshed

a5ebef7

Oops errant prints

7bba610

sixolet mentioned this pull request May 1, 2017

Release mypy 0.510 #3273

Closed

Merge branch 'master' into decorated-type

5b927ac

JukkaL reviewed May 2, 2017

View reviewed changes

sixolet added 3 commits May 2, 2017 14:17

More tests that Jukka asked for

19a0964

Newline at end of file

8c803da

Merge master

1c8b1a9

gvanrossum added the topic-pep-484 label May 3, 2017

JukkaL mentioned this pull request May 4, 2017

Support function decorators excellently #3157

Closed

9 tasks

sixolet added 2 commits June 14, 2017 21:15

Merge branch 'master' into decorated-type

4e3c442

Change decorated_type to declared_type

145e386

sixolet changed the title ~~Support for decorated_type, to give types to decorated functions~~ Support for declared_type, to give types to decorated functions Jun 15, 2017

JukkaL self-assigned this Jun 30, 2017

This was referenced Aug 23, 2017

Declare and use attributes of function objects #2087

Closed

Ability to use Callable type alias when annotating functions #1641

Closed

JukkaL reviewed Sep 20, 2017

View reviewed changes

ilevkivskyi reviewed Sep 20, 2017

View reviewed changes

ilevkivskyi mentioned this pull request Oct 29, 2017

Decorator syntax for type casting in decorators python/typing#490

Closed

JukkaL closed this Apr 4, 2018

OlegAlexander mentioned this pull request Jun 26, 2023

Ability to use Callable type alias when annotating functions python/typing#1419

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support for `declared_type`, to give types to decorated functions #3291

Support for `declared_type`, to give types to decorated functions #3291

sixolet commented May 1, 2017

JukkaL left a comment

JukkaL May 2, 2017

JukkaL May 2, 2017

JukkaL May 2, 2017

JukkaL May 2, 2017

JukkaL May 2, 2017

JukkaL May 2, 2017

JukkaL commented May 2, 2017

sixolet commented Jun 15, 2017

gvanrossum commented Jun 20, 2017

gvanrossum commented Sep 19, 2017

JukkaL left a comment

JukkaL Jun 21, 2017

JukkaL Jun 21, 2017

JukkaL Jun 21, 2017

JukkaL Sep 20, 2017

JukkaL Sep 20, 2017

JukkaL Sep 20, 2017

JukkaL Sep 20, 2017

JukkaL Sep 20, 2017

JukkaL Sep 20, 2017

ilevkivskyi Sep 20, 2017

JukkaL Sep 20, 2017

ilevkivskyi Sep 20, 2017

JukkaL Sep 20, 2017

ilevkivskyi Sep 20, 2017

JukkaL commented Apr 4, 2018


		def dec(f: T) -> T: pass

		@decorated_type(Callable[[int], str]) # E: Incompatible types (inferred decorated type Callable[[str], str], declared decorated type Callable[[int], str])

		y: int = foo.f(1) # E: Incompatible types in assignment (expression has type "str", variable has type "int")

		reveal_type(foo.f) # E: Revealed type is 'def (builtins.int) -> builtins.str'

Support for declared_type, to give types to decorated functions #3291

Support for declared_type, to give types to decorated functions #3291

Conversation

sixolet commented May 1, 2017

JukkaL left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JukkaL commented May 2, 2017

sixolet commented Jun 15, 2017

gvanrossum commented Jun 20, 2017

gvanrossum commented Sep 19, 2017

JukkaL left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

JukkaL commented Apr 4, 2018

Support for `declared_type`, to give types to decorated functions #3291

Support for `declared_type`, to give types to decorated functions #3291