Clarification - protocol properties vs attributes

[erichulburd]

Describe the Bug

When implementing a typing.Protocol it's not entirely clear to me when an attribute conforms to a property protocol method and vice versa. In particular, I've noticed that Pyrefly will allow typing.Protocol properties to be implemented with attributes, but not vice versa; Pyright somewhat contradictorily, permits properties to be implemented with attributes at a call site, but not if the protocol is inherited.

The Pyrefly behavior is more self-consistent, but I would like to confirm why this is the case. Is there official Python documentation to substantiate this behavior? If not, then why does Pyrefly accept attributes as protocol property implementations but not vice versa?

Note, the PEP 544, which introduced Protocols, explicitly ruled in protocol variable members (what I've referred to as attributes above): https://peps.python.org/pep-0544/#allow-only-protocol-methods-and-force-use-of-getters-and-setters

Examples

Protocol inheritance

from typing import List, Protocol
from dataclasses import dataclass


class Example1(Protocol):
    @property
    def samples(self) -> List[int]: ...


@dataclass(kw_only=True)
class Example1Impl(Example1):
    samples: List[int]


def _example1(result: Example1) -> List[int]:
    return result.samples


class Example2(Protocol):
    samples: List[int]


@dataclass(kw_only=True)
class Example2Impl(Example2):
    @property
    def samples(self) -> List[int]:
        return [1, 2, 3]

def _example2(result: Example2) -> List[int]:
    return result.samples


def main():
    r1 = Example1Impl(samples=[4, 5, 6])
    _example1(r1)

    r2 = Example2Impl()
    _example2(r2)

Pyrefly:

  --> src/my_project/example.py:26:9
   |
26 |     def samples(self) -> List[int]:
   |         ^^^^^^^
   |
  `Example2Impl.samples` is a property, but `Example2.samples` is not

Pyright:

src/my_project/example.py
  src/my_project/example.py:12:5 - error: "samples" incorrectly overrides property of same name in class "Example1" (reportIncompatibleMethodOverride)
  src/my_project/example.py:26:9 - error: "samples" overrides symbol of same name in class "Example2"
    "property" is not assignable to "List[int]" (reportIncompatibleVariableOverride)
2 errors, 0 warnings, 0 informations

Protocol at call site

from typing import List, Protocol
from dataclasses import dataclass


class Example1(Protocol):
    @property
    def samples(self) -> List[int]: ...


@dataclass(kw_only=True)
class Example1Impl:
    samples: List[int]


def _example1(result: Example1) -> List[int]:
    return result.samples


class Example2(Protocol):
    samples: List[int]


@dataclass(kw_only=True)
class Example2Impl:
    @property
    def samples(self) -> List[int]:
        return [1, 2, 3]

def _example2(result: Example2) -> List[int]:
    return result.samples


def main():
    r1 = Example1Impl(samples=[4, 5, 6])
    _example1(r1)

    r2 = Example2Impl()
    _example2(r2)

Pyrefly:

ERROR Argument `Example2Impl` is not assignable to parameter `result` with type `Example2` in function `_example2` [bad-argument-type]
  --> src/my_project/example.py:38:15
   |
38 |     _example2(r2)

Pyright:

  src/my_project/example.py:38:15 - error: Argument of type "Example2Impl" cannot be assigned to parameter "result" of type "Example2" in function "_example2"
    "Example2Impl" is incompatible with protocol "Example2"
      "samples" is invariant because it is mutable
      "samples" is an incompatible type
        "property" is not assignable to "List[int]" (reportArgumentType)
1 error, 0 warnings, 0 informations

Sandbox Link

https://pyrefly.org/sandbox/?project=N4IgZglgNgpgziAXKOBDAdgEwEYHsAeAdAA4CeS4ATrgLYAEALqcROgOZ0Q3G6UN0AZCHAYAaOgAVqDXAGNcUADrow1eplQNUsqKjhx4nbr34atOvXGXX0F-XQCi%2BVN1gBGABRTcM%2BVACUiMp0IXQAAsTUxDB8pMGhmDBgdGiu8B4GUGD%2BdAC0AHyCwgwA2qwMALqIdIS1NsphZtq6%2Bh4A1gDuAPq46FCkALwAKpQArjD%2BynZwjs5pbgCSrh5OLsTugfEhqevw1UIiZeiV9VhJdF0wc7uelPCjUAzVq-M5BUWH5VVbdHcMo5R0L97o9CDtYFZ0DZprM1rAAExeaRyBSbIGhcF7D6lL6nRqaZqWdrdXr9YZjCZTFozF67eFLdYra4ItGhcKRXDRWI-RLJTFwDIwLJvQoHHHHb7otkhP4AoElNzieHiADMFRsvIuVzhMERdzgDyesLS8JF2KOlSCUuB-0BwINoP5p01NFQrA8rNClDcdAGxpuDKgGR1cAGJQALOIAKziABsFUm1suzJgtzciZ%2BlHhvv9CMDHp%2ByZ1etNNhAohAowY0DgJHIiBAAGI6ABVatQCBMOhgUa2au9SHKTVgXiuhhddCjGjYGIefDVcpmkSUK1s2V2sCKEAAOSnM5XdGA%2BAAvlvlOWQGQ7mB%2BoQZDQoBRmxJSNf%2BikMDgCHR5CoIGwAU0CBekIZRmwAZRgGA6AACwYBhiDgRAAHpkKvJJb14NhkJgdBkMwOQ4GQ39IAAyggN6ZDu14OhUAANzdXRsFgH9elIwD%2ByBTlONrZQyAYGDelyOiYjgYCgT9LcVUINx4TPdAQGPCttGrESADFoBgCg0CwPAiDIRSgA

(Only applicable for extension issues) IDE Information

No response

[erichulburd]

Some useful references on this topic:

• https://discuss.python.org/t/typing-spec-gap-about-properties-in-protocols/87243
• Can an attribute be overridden by a property or vice versa? microsoft/pyright#10291
• Mypy disallows overriding an attribute with a property python/mypy#4125
• https://peps.python.org/pep-0767/#protocols
• https://peps.python.org/pep-0767/#clarifying-interaction-of-property-and-protocols

[samwgoldman]

Thanks for the question! I remember we discussed this quite a bit when implementing structural subtyping for attributes.

Overriding a base class property with an attribute, or vice versa, is always unsafe because accessing a property from the class object gives back the property itself. We should have that type[B] <: type[A] if B is a subclass of A, so this breaks substitutability. That's why Pyright rejects the code, I believe. I found this comment from @erictraut. Example

class A:
  @property
  def x(self) -> int: ...

class B(A):
  x: int

reveal_type(A.x) # property
reveal_type(B.x) # int

Forgetting the class object access behavior for a moment, overriding a read-write attribute with a read-only property is unsafe because the subclass has more capabilities. The subclass can read and write, but the base class can only read. Example:

class A:
    x: int

class B(A):
    @property
    def x(self) -> int: ...

def test(o: A):
    o.x = 0

test(B()) # boom

We could have decided to allow properties with both getters and setters to override an attribute in the base class. This would still be unsafe because of class object access, but we are already unsafe in this way because we allow attributes to override properties, so I don't think it's meaningfully worse.

We might also want to consider the del behavior. An attribute can be deleted, but you would need to define x.deleter for every property that overrides an attribute, and that would probably feel very onerous, since del is already very uncommon.

I'm certainly open to changing our behavior here -- do you have some "real" code you are trying to write? Or is this more of a theoretical question? In any event, I wonder if this isn't a better topic for the Python Typing discussion forum: https://discuss.python.org/c/typing/32

[erichulburd]

I think you're right that this is best settled in https://discuss.python.org/c/typing/32. I will give it a shot; just want to suss out your stance a bit more, first.

I definitely have real code in mind, but I can't share it at this point, unfortunately (it will be eventually open sourced). Suffice it to say, I am writing up a styleguide that intends to provide some clarity around inheritance and protocol handling. I anticipate that the patterns will become ubiquitous in our code base, so I want to avoid any patterns that are less well specified in the official PEPs.

That said, because Pyright and Pyrefly agree that Example2Impl implements Example2 in the "Protocol at call site" example in the PR description, I'm almost willing to take the leap of faith without explicit direction in the PEPs (I would additionally advocate marking all of our protocols as @final to avoid all of the pitfalls of inheriting protocols completely).

I think my ask of Pyrefly would be to make an explicit statement on the following:

1. Source of ambiguity: Protocols allow attributes, but there is no specific guidance on whether and how properties can implement attributes (and vice versa).
2. Pyrefly's strong views: I can't speak for you here, but a statement along the following lines would be helpful. Pyrefly treats protocol attributes as having property getter, setter, deleter, and corresponding kwarg in the initializer. Although this is not well documented it in any PEP, see corresponding discussion in Python typing and draft PEP, which we (along with Pyright, Astral, etc) are putting our weight behind.
3. Pyrefly's lighter views: Again, I can't speak for you here, but something along the following lines would be helpful.

• Because we treat class A with attribute x as an implementation of protocol P with property x, and type(A.x) != type(P.x) at runtime, Pyrefly treats P.x as undefined.

I don't mean to put all of this on you, but if I can get a sense of your openness to these ideas, I would be happy to contribute content.