Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

General support for properties #220

Open
JukkaL opened this issue Jul 13, 2013 · 26 comments

Comments

@JukkaL
Copy link
Collaborator

@JukkaL JukkaL commented Jul 13, 2013

Add support for type checking properties.

We should initially support at least the decorator syntax for read-only properties and the assignment syntax for both read-only and read-write properties, as these seem to be the most frequently used variants:

class A:
    @property
    def foo(self) -> int: return 2

    def get_bar(self) -> int: return 3
    def set_bar(self, value: int) -> None: ...
    bar = property(get_bar, set_bar)

Also support defining a deleter and a docstring, at least when using the call syntax.

@JukkaL

This comment has been minimized.

Copy link
Collaborator Author

@JukkaL JukkaL commented Jul 14, 2013

Generic descriptor support would also let us support properties, but the distinction between read-only and settable properties could be lost. A property would have type property[ObjT, ValT], where ObjT is the class that defines the property and ValT is the type of the property value.

In order to properly model settable properties, we should have something like property[ObjT, GetT, SetT], and allow SetT to be undefined. However, there is currently no way of letting the value of a type argument be undefined. We could also generally support default values for type variables, and undefined would a valid default value.

@JukkaL

This comment has been minimized.

Copy link
Collaborator Author

@JukkaL JukkaL commented Sep 21, 2013

Read-only properties using @property work, but not the other forms.

@ghost ghost assigned JukkaL Dec 24, 2013
@JukkaL JukkaL added the hackmypy label Jul 15, 2014
@JukkaL JukkaL removed front end labels Jul 25, 2014
@JukkaL JukkaL removed their assignment Jul 25, 2014
@JukkaL JukkaL added the priority label Oct 8, 2014
@thomascellerier

This comment has been minimized.

Copy link

@thomascellerier thomascellerier commented Mar 31, 2015

Any progress on this? I am using mypy to check some new production (yes i know :)) python daemon and it worked fine until I started using properties. Do you know of any workaround? Even using the old properties notation i get issues with setters.
A property cannot be overloaded
'overload' decorator expected

Thanks for this great tool.

@gvanrossum

This comment has been minimized.

Copy link
Member

@gvanrossum gvanrossum commented Mar 31, 2015

I'm guessing you're talking about an example like this:

class C:
    def __init__(self):
        self._x = None

    @property
    def x(self):
        """I'm the 'x' property."""
        return self._x

    @x.setter
    def x(self, value):
        self._x = value

    @x.deleter
    def x(self):
        del self._x

Indeed mypy goes crazy over this:

a.py, line 5: A property cannot be overloaded
a.py, line 5: 'overload' decorator expected
a.py, line 10: Name 'x' is not defined
a.py, line 10: 'overload' decorator expected
a.py, line 14: Name 'x' is not defined
a.py, line 14: 'overload' decorator expected
@JukkaL

This comment has been minimized.

Copy link
Collaborator Author

@JukkaL JukkaL commented Apr 5, 2015

Yeah, this hasn't been implemented yet. It's been on my to-do list for a long time, but since the code I've worked with haven't used setters much I've never gotten around to actually implementing this. The implementation should not be very difficult, though.

I'll try to get this working reasonably soon (though probably only after PyCon, i.e. after April 15th).

@JukkaL

This comment has been minimized.

Copy link
Collaborator Author

@JukkaL JukkaL commented Apr 5, 2015

I just pushed experimental setter support. The following code now works as expected:

class C:
    def __init__(self) -> None:
        self._x = 0

    @property
    def x(self) -> int:
        return self._x

    @x.setter
    def x(self, value: int) -> None:
        self._x = value

    @x.deleter
    def x(self) -> None:
        del self._x

c = C()
c.x = 2 * c.x
c.x = ''   # Error!

The implementation still needs refinement. For example, x = property(...) is not supported.

Let me know if/when you encounter additional issues.

@thomascellerier

This comment has been minimized.

Copy link

@thomascellerier thomascellerier commented Apr 8, 2015

Thanks, the file containing the properties now typechecks.
However I have another file in the same module assigning the property to a variable which fails to typecheck with this error (given a property called x with a getter and setter):
a.py

class C:
    @property
    def x(self) -> Dict[str, Any]:
        return self._x

    @x.setter
    def x(self, value: Dict[str, Any]) -> None:
        self._x = value

b.py

def foo(c: a.C):
    x = c.x

Cannot determine type of 'x'.

@JukkaL

This comment has been minimized.

Copy link
Collaborator Author

@JukkaL JukkaL commented Apr 10, 2015

Do you have cyclic imports? Mypy has trouble with them (#481) in general. However, it should be pretty easy to fix mypy to support your example -- assuming that this actually is due to cyclic imports.

@NYKevin NYKevin mentioned this issue Jan 20, 2016
@JukkaL

This comment has been minimized.

Copy link
Collaborator Author

@JukkaL JukkaL commented Feb 3, 2016

This still doesn't work (reported by Tim Abbott):

class C:
    def __init__(self) -> None:
        self._x = 0

    @property
    def x(self) -> int:
        return self._x

    @x.setter
    def set_x(self, value: int) -> None:   # Note the name of the method! "def x" works.
        self._x = value
@JukkaL

This comment has been minimized.

Copy link
Collaborator Author

@JukkaL JukkaL commented Feb 3, 2016

The last example seems to be only valid in Python 2, not Python 3.

@ddfisher ddfisher added this to the 0.4.0 milestone Mar 2, 2016
@gnprice gnprice removed the priority label Mar 2, 2016
@Deimos

This comment has been minimized.

Copy link
Contributor

@Deimos Deimos commented Mar 10, 2017

Here's some more examples of strange property-related behavior:

This code produces a mypy error as it should (the passwordconstructor argument was "accidentally" annotated as an int, but the property setter expects a str value):

class User:
    @property
    def password(self) -> str:
        raise AttributeError('Password is write-only')

    @property.setter
    def password(self, value: str) -> None:
        # TODO: add hashing
        self.password_hash = value

    def __init__(self, username: str, password: int) -> None:
        self.username = username
        self.password = password

Running mypy gives mypy_test.py:13: error: Incompatible types in assignment (expression has type "int", variable has type "str") as expected.

However, if I just rearrange the order of the methods so that the constructor is first and the property methods come after it, there's no error produced any more:

class User:
    def __init__(self, username: str, password: int) -> None:
        self.username = username
        self.password = password

    @property
    def password(self) -> str:
        raise AttributeError('Password is write-only')

    @property.setter
    def password(self, value: str) -> None:
        # TODO: add hashing
        self.password_hash = value

Running mypy on that code has no output.

In addition, even the "more correct" behavior from the original code was only because the property was annotated incorrectly. Because attempting to read .password always raises an error, the correct way to write this code should have been (annotation on the password constructor argument "fixed" to str as well):

from mypy_extensions import NoReturn

class User:
    @property
    def password(self) -> NoReturn:
        raise AttributeError('Password is write-only')

    @property.setter
    def password(self, value: str) -> None:
        # TODO: add hashing
        self.password_hash = value

    def __init__(self, username: str, password: str) -> None:
        self.username = username
        self.password = password

However, even though this code should now be correct, mypy produces mypy_test.py:15: error: Incompatible types in assignment (expression has type "str", variable has type NoReturn).

So it seems like mypy is using the return value of the "getter" to determine the value the "setter" can accept, but that won't necessarily always match up.

@gvanrossum

This comment has been minimized.

Copy link
Member

@gvanrossum gvanrossum commented Mar 10, 2017

Hm, I think we ran into a similar issue (about potential discrepancies between getter and setter) for descriptors (#2266).

@funkyfuture

This comment has been minimized.

Copy link

@funkyfuture funkyfuture commented Mar 15, 2017

edit: previously contained #3011

@gvanrossum

This comment has been minimized.

Copy link
Member

@gvanrossum gvanrossum commented Mar 15, 2017

it'd be nice to support abc.abstractproperty as well.

Should already be supported, see #263.

@gvanrossum

This comment has been minimized.

Copy link
Member

@gvanrossum gvanrossum commented Mar 15, 2017

@funkyfuture Please file a separate issue.

@gvanrossum gvanrossum removed this from the 0.5 milestone Mar 29, 2017
@douglas-treadwell

This comment has been minimized.

Copy link

@douglas-treadwell douglas-treadwell commented Jul 11, 2017

@JukkaL

So I've seen this commit a103f25 that closed #263. It seems to check for exactly "property" or "abc.abstractproperty" for property support.

Is there a way to support non-stdlib property decorators such as SqlAlchemy's hybrid_property, which has getter, setter, deleter, as well as expression and comparator?

I'd rather not need to litter my code with # type: ignore.

@douglas-treadwell

This comment has been minimized.

Copy link

@douglas-treadwell douglas-treadwell commented Jul 11, 2017

Also, I'd be happy to fix this myself if someone @JukkaL can give me enough context to do so in less time than it'd take just to fix it themselves.

@gvanrossum

This comment has been minimized.

Copy link
Member

@gvanrossum gvanrossum commented Jul 11, 2017

We've got descriptor support for a while now, maybe it will be easy to build on top of this. But please don't expect a lot of hand-holding (the core mypy team is currently stretched to the max).

@douglas-treadwell

This comment has been minimized.

Copy link

@douglas-treadwell douglas-treadwell commented Jul 11, 2017

For anyone else who comes across this thread, Descriptor support was discussed in #244 and #2266, and a solution for Descriptors was merged in the latter. I'm investigating whether I can use this to resolve my issue.

@srittau

This comment has been minimized.

Copy link
Contributor

@srittau srittau commented Nov 12, 2017

It also seems that the getter and setter need to have the same type at the moment:

class Foo:
    @property
    def bar(self) -> str: ...
    @bar.setter
    def bar(self, v: int) -> None: ...

Foo().bar = 3

This will cause mypy 0.550 to complain:

foo.py:7: error: Incompatible types in assignment (expression has type "int", variable has type "str")

While this example is a bit silly, I have a real-world example involving setting arbitrary Mappings and getting a custom Mapping subtype.

@gvanrossum

This comment has been minimized.

Copy link
Member

@gvanrossum gvanrossum commented Nov 12, 2017

@srittau

This comment has been minimized.

Copy link
Contributor

@srittau srittau commented Nov 12, 2017

Actually my last comment was already separately filed as issue #4167.

@ilevkivskyi

This comment has been minimized.

Copy link
Collaborator

@ilevkivskyi ilevkivskyi commented Dec 10, 2018

Another thing that doesn't work came in #6045: accessing attributes of a property object in the class body (for example fset, fget etc). This can be probably partially solved by some special casing in checkmember.py.

nbraud added a commit to nbraud/ppb-vector that referenced this issue Dec 15, 2018
mypy cannot currently cope with the attributes of properties, incl. fget:

  python/mypy#220
@nbraud nbraud mentioned this issue Dec 15, 2018
2 of 2 tasks complete
@Deimos

This comment has been minimized.

Copy link
Contributor

@Deimos Deimos commented Jun 22, 2019

In my comment above from March 9, 2017, I noted that putting __init__ before property getter/setters caused mypy to miss a (false-positive) error that it otherwise reported.

I don't know if it's useful information at all, but I just wanted to point out that the new semantic analyzer no longer has that particular issue, and it will report the same error (which isn't actually one) regardless of whether __init__ is at the top or bottom of the class.

Specifically, here's the code:

from mypy_extensions import NoReturn

class User:
    def __init__(self, username: str, password: str) -> None:
        self.username = username
        self.password = password

    @property
    def password(self) -> NoReturn:
        raise AttributeError('Password is write-only')

    @property.setter
    def password(self, value: str) -> None:
        # TODO: add hashing
        self.password_hash = value

And the output from both analyzers:

$ mypy test_mypy.py
$ mypy --new-semantic-analyzer test_mypy.py
test_mypy.py:6: error: Incompatible types in assignment (expression has type "str", variable has type "NoReturn")
@vitodsk

This comment was marked as outdated.

Copy link

@vitodsk vitodsk commented Jul 18, 2019

should inference work also with custom types ?
for example:

class MyType(enum.Enum):

      test_1 = {
             "key_a": 12,
             "key_b": 23
      }

      test_2 = {
           "key_a": 10,
           "key_b": 9
     }

     def __init__(self, vals):
        self.key_a = vals['key_a']
        self.key_b = vals['key_b']
class SomeClass:
     def __init__():
          self.__my_enum_property = None

     @property
     def my_enum_property(self):
         return self.__my_enum_property

     @my_enum_property.setter
     def my_enum_property(self, val):
         self.__my_enum_property = val

def some_foo():
    obj = SomeClass()
    obj.my_enum_property = MyType.test_1

I get as error, for each line where MyType.test_* is assigned:

error: Property "my_enum_property" defined in "SomeClass" is read-only

where am I doing it wrong ?

@gonzaloamadio

This comment has been minimized.

Copy link

@gonzaloamadio gonzaloamadio commented Nov 4, 2019

I have this file


import operator
from functools import total_ordering, wraps

def _time_required(f):
    @wraps(f)
    def wrapper(self, other):
        if not isinstance(other, Time):
            raise TypeError("Can only operate on Time objects.")
        return f(self, other)
    return wrapper
def _int_required(f):
    @wraps(f)
    def wrapper(self, value):
        if not isinstance(value, int):
            raise TypeError("An integer is required.")
        return f(self, value)
    return wrapper
def _balance(a, b):
    if b >= 0:
        while b >= 60:
            a += 1
            b -= 60
    elif b < 0:
        while b < 0:
            a -= 1
            b += 60
    return a, b

@total_ordering
class Time:

    def __init__(self, h: int = 0, m: int = 0, s: int = 0):
        self._h = h
        self._m = m
        self._s = s

    @property
    def h(self):
        return self._h

    @h.setter
    @_int_required
    def h(self, value):
        self._h = value

    @property
    def m(self):
        return self._m

    @m.setter
    @_int_required
    def m(self, value):
        self._m = value
        self._h, self._m = _balance(self._h, self._m)

    @property
    def s(self):
        return self._s

    @s.setter
    @_int_required
    def s(self, value):
        self._s = value
        self._m, self._s = _balance(self._m, self._s)

    def _operation(self, other, method):
        h = method(self.h, other.h)
        m = method(self.m, other.m)
        s = method(self.s, other.s)
        return Time(h, m, s)

    @_time_required
    def __add__(self, other):
        return self._operation(other, operator.add)

    @_time_required
    def __sub__(self, other):
        return self._operation(other, operator.sub)

    @_time_required
    def __eq__(self, other):
        return (self.h == other.h and
                self.m == other.m and
                self.s == other.s)

    @_time_required
    def __lt__(self, other):
        if self.h < other.h:
            return True
        if self.h > other.h:
            return False
        if self.m < other.m:
            return True
        if self.m > other.m:
            return False
        return self.s < other.s

    def __repr__(self):
        return f"<Time {self.h:02}:{self.m:02}:{self.s:02}>"

if __name__ == '__main__':
    a = Time(14, 23, 10)
    try:
        a.h = "Hello, world!"
    except Exception as err:
        print(err)

And when I run mypy, I have the following errors.

class.py:47: error: Decorated property not supported     **<-- Line @h.setter**
class.py:56: error: Decorated property not supported     **<-- Line @m.settet**
class.py:66: error: Decorated property not supported     **<-- Line @s.setter**
class.py:110: error: Property "h" defined in "Time" is read-only  **<-- Line:  a.h = "Hello, world!"**

If I comment out the decorators under the setters, mypy check runs OK. No errors

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.