`self` annotation not checked in `__index__`?

[MarcoGorelli]

Bug Report

To Reproduce

from __future__ import annotations
from typing import Callable, TypeVar, Any, Protocol, Self, Any

T = TypeVar('T', covariant=True)

class Array(Protocol[T]):
    def __index__(self: Array[int], /) -> int: ...
    def dtype(self) -> T: ...
    def __rmul__(self, other: Any) -> Self: ...

def func(a: Array[str], b: list[bool]) -> None:
    result = b * a
    reveal_type(result)

https://mypy-play.net/?mypy=latest&python=3.12&gist=596075fa56c1bd714ec44b224f5ed97c

Expected Behavior

t.py:13: note: Revealed type is "Array[str]"

(which is what Pyright does)

This is because list.__mul__ is defined as

https://github.com/python/typeshed/blob/7aad2554355b55e70e4137fadf48f294ee79efaf/stdlib/builtins.pyi#L1124

    def __mul__(self, value: SupportsIndex, /) -> list[_T]: ...

and SupportsIndex checks for __index__(self) -> int, and if value is Array[str], then Array.__index__ is not matched

Actual Behavior

t.py:13: note: Revealed type is "builtins.list[builtins.bool]"

Your Environment

• Mypy version used: 1.16.1
• Mypy command-line flags:
• Mypy configuration options from mypy.ini (and other config files):
• Python version used: 3.12

[sterliakov]

This is not specific to __index__ but rather is a glaring hole in protocol compatibility checking. Here's a conceptually simpler repro without typeshed dependency:

from __future__ import annotations
from typing import Generic, Protocol, TypeVar

_T = TypeVar("_T")

class SupportsFoo(Protocol):
    def foo(self) -> None: ...
    
class Bad(Generic[_T]):
    x: _T
    
    def foo(self: Bad[int]) -> None: ...
    
bad: Bad[str] = Bad()
fail: SupportsFoo = bad  # mypy: OK

The problem is that failed selftype checks only produce a diagnostic but still return the original signature. All diagnostics are silenced when checking protocol compatibility, so this error remains in mypy internals and never makes it to mypy output.

[yangdanny97]

It's not just protocols, I think. If we change SupportsFoo to be a concrete class and extend it with Bad, there is still no error:

from __future__ import annotations
from typing import Generic, Protocol, TypeVar

_T = TypeVar("_T")

class SupportsFoo():
    def foo(self) -> None: ...
    
class Bad(Generic[_T], SupportsFoo):
    x: _T
    
    def foo(self: Bad[int]) -> None: ...
    
bad: Bad[str] = Bad()
fail: SupportsFoo = bad  # mypy: OK

[sterliakov]

@yangdanny97 surprisingly, that's a different issue located in another part of the implementation, they do not share the same root cause (unless I'm missing some high-level connection, of course, but they really do not rely on the same helpers). Your snippet represents a problem with checking override validity after binding self (node_type_from_base in checker.py binds self, and override check sees def foo() -> None in both superclass and subclass). Binding itself succeeds there, please consider reporting this problem separately for better inventory management.

However, they both share a common issue: things get heated when overloads come into play (still not too hard when checking a concrete type against protocol, but worse hen checking overrides). And even without overloads, how should narrowing selftype behave? Note that we can't just compare callables before binding self, as that would mean any override violates LSP (and it does, Child.foo(Parent()) is unsafe, but we probably don't want to enforce that). So the first argument should undergo special treatment, depending on... what, exactly? The following makes some sense to me for non-overloads, but I'm not very certain:

• Let's assume we're checking foo definition overridden in class Child that inherits its original implementation from Parent. That means Parent is in Child MRO directly, no structural types today.
• Extract self argument type from Child.foo (let's call it B) and Parent.foo (A).
• Map B to supertype A. If any arguments were ignored (see example below) during this transformation and they aren't wider than their widest possible values, declare override incompatible and exit. Otherwise, call the mapped result B' and proceed. I do not yet understand how to express this precisely, just a braindump - how do we identify such "extra" args in general case?
• Check if B' is a proper subtype of A. If it is not, declare override incompatible and exit.
• Bind self and check the rest of the signature (no changes here). Again, if incompatible, fail and exit.
• Declare override compatible and stop.

"Lossy mapping" example (should be disallowed as any A instance can run a.foo() but the same isn't true for an arbitrary b):

from typing import Generic, TypeVar

T = TypeVar("T")
U = TypeVar("U")

class A(Generic[T]):
    x: T
    def foo(self) -> None: ...

class B(Generic[T, U], A[T]):
    y: U
    def foo(self: B[T, str]) -> None: ...

"Lossy mapping" example that should be allowed:

from typing import Generic, TypeVar

T = TypeVar("T")
U = TypeVar("U", bound=str, covariant=True)

class A(Generic[T]):
    x: T
    def foo(self) -> None: ...

class B(Generic[T, U], A[T]):
    y: U
    def foo(self: B[T, str]) -> None: ...

...ough, and I haven't even mentioned protocols and overloads. This probably needs a spec or someone with understanding deep enough to lay out the core rules for such assignability.