Fix overload resolution for type aliases to Any

[pablogsal]

Aliases like Incomplete: TypeAlias = Any were being downgraded to TypeOfAny.special_form, the same flavor used for NewType-style higher-kinded types. has_any_type ignores special_form Anys when checking for overload ambiguity, which meant a 2-arg call to os.environ.get with an alias-Any default would silently match the default: None = None overload and return str | None instead of Any. No bueno

Add a dedicated TypeOfAny.from_alias_target flavor that suppresses --disallow-any-explicit at use sites (the original reason for the downgrade) but still counts as a real Any everywhere else. Stats reporting and the Literal[...] arg check are extended to keep their existing behavior for the new flavor.

Fixes #21344

[hauntsaninja]

Thank you! This makes some amount of sense and I dislike how mypy's current behaviour seemingly breaks referential transparency pretty badly

But this is a lot of primer diff and would be disruptive for users, so I think we might have to explore a few other options or additional changes

[pablogsal]

Thank you! This makes some amount of sense and I dislike how mypy's current behaviour seemingly breaks referential transparency pretty badly

But this is a lot of primer diff and would be disruptive for users, so I think we might have to explore a few other options or additional changes

I don't have a strong opinion here so happy to work on whatever direction you think makes sense :)

[Hnasar]

Thanks for this fix -- it seems more correct! And I agree, much of the primer diff looks reasonable, like the trio changes, and the new unused-ignores, but I'm concerned with the numpy and pandas regressions which seem like they could be related to Any-in-Generics?

E.g.

static-frame (https://github.com/static-frame/static-frame)
+ static_frame/core/util.py:1807: error: Returning Any from function declared to return "ndarray[Any, Any]"  [no-any-return]

Looking https://github.com/static-frame/static-frame/blob/master/static_frame/core/util.py#L1807 is returning values.T where values: TNDArrayAny and TNDArrayAny = np.ndarray[tp.Any, tp.Any].

And here's the (reasonable) definition of ndarray.T

So I'm not sure where this regression is coming from. Perhaps we need to update some usage of TypeOfAny.special_form to account for the new TypeOfAny.from_alias_target?

(Perhaps the regression in this numpy/pandas usage could be demonstrated in a test to narrow (ha!) it down?)

[Hnasar]

nit: name is now incongruous with body. suggest: is_special_form_or_alias_any

[hauntsaninja]

I was playing around with a simpler diff that looked like:

diff --git a/mypy/semanal.py b/mypy/semanal.py
index f9b52a0dc..664255000 100644
--- a/mypy/semanal.py
+++ b/mypy/semanal.py
@@ -8269,7 +8269,7 @@ def make_any_non_explicit(t: Type) -> Type:
 class MakeAnyNonExplicit(TrivialSyntheticTypeTranslator):
     def visit_any(self, t: AnyType) -> Type:
         if t.type_of_any == TypeOfAny.explicit:
-            return t.copy_modified(TypeOfAny.special_form)
+            return t.copy_modified(TypeOfAny.from_another_any, original_any=t)
         return t
 
     def visit_type_alias_type(self, t: TypeAliasType) -> Type:

Seemed pretty similar to this PR. In particular, it also hit static_frame/core/util.py:1807 which I think to your question implies that there isn't some usage of TypeOfAny.special_form that could be updated in this diff to avoid it. So still something to be understood!

[pablogsal]

I was playing around with a simpler diff that looked like:

diff --git a/mypy/semanal.py b/mypy/semanal.py
index f9b52a0dc..664255000 100644
--- a/mypy/semanal.py
+++ b/mypy/semanal.py
@@ -8269,7 +8269,7 @@ def make_any_non_explicit(t: Type) -> Type:
 class MakeAnyNonExplicit(TrivialSyntheticTypeTranslator):
     def visit_any(self, t: AnyType) -> Type:
         if t.type_of_any == TypeOfAny.explicit:
-            return t.copy_modified(TypeOfAny.special_form)
+            return t.copy_modified(TypeOfAny.from_another_any, original_any=t)
         return t
 
     def visit_type_alias_type(self, t: TypeAliasType) -> Type:

Seemed pretty similar to this PR. In particular, it also hit static_frame/core/util.py:1807 which I think to your question implies that there isn't some usage of TypeOfAny.special_form that could be updated in this diff to avoid it. So still something to be understood!

Thanks for checking! I can investigate more tomorrow indeed

[pablogsal]

I investigated the pandas-stubs / numpy-ish primer regressions. I think the issue is that this change affects aliases involving tuple[Any, ...] / np.ndarray[...]. Previously those nested Anys were special_form, so has_any_type() ignored them for overload ambiguity. With this PR they become “real” Anys, so overload resolution enters the ambiguity path much more often.

For pandas-stubs operator overloads, that means calls that used to infer Series[Any] / Index[Any] now match multiple overloads with different return types and fall back to bare Any, which explains errors like:

  Expression is of type "Any", not "Series[Any]"
  Expression is of type "Any", not "Index[Any]"

A small reduced version is:

 from typing import Any, Generic, TypeAlias, TypeVar, overload, Sequence
 from typing_extensions import Never

 S = TypeVar("S")
 Sh = TypeVar("Sh")

 A: TypeAlias = Any

 class Array(Generic[Sh, S]): pass

 Shape: TypeAlias = tuple[Any, ...]
 ArrBool: TypeAlias = Array[Shape, bool]

 class Series(Generic[S]):
     @overload
     def __add__(self: Series[Never], other: complex | Sequence[Any]) -> Series[Any]: ...
     @overload
     def __add__(self: Series[S], other: Array[Any, S]) -> Series[S]: ...
     @overload
     def __add__(self: Series[S], other: Array[Any, bool]) -> Series[S]: ...
     @overload
     def __add__(self: Series[bool], other: Array[Any, bool]) -> Series[bool]: ...
     @overload
     def __add__(self: Series[str], other: Array[Any, bool]) -> Never: ...
     def __add__(self, other): ...

 def f(x: Series[A], y: ArrBool):
     reveal_type(x + y)

On master, this reveals Series[Any]. On this branch, it reveals bare Any.

So I think the fix direction is to preserve the new “real Any” behavior only for aliases whose target is exactly Any / another alias to exactly Any, while keeping nested explicit Anys inside larger alias targets as special_form for the old overload-ambiguity behavior.

[pablogsal]

Will take a look at the primer later to see what's going on with these other changes

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

colour (https://github.com/colour-science/colour)
- colour/utilities/metrics.py:88: error: Incompatible return value type (got "floating[_16Bit] | floating[_32Bit] | float64", expected "ndarray[tuple[Any, ...], dtype[floating[_16Bit] | floating[_32Bit] | float64]]")  [return-value]

[pablogsal]

I was playing around with a simpler diff that looked like:

Diff from mypy_primer, showing the effect of this PR on open source code:

colour (https://github.com/colour-science/colour)
- colour/utilities/metrics.py:88: error: Incompatible return value type (got "floating[_16Bit] | floating[_32Bit] | float64", expected "ndarray[tuple[Any, ...], dtype[floating[_16Bit] | floating[_32Bit] | float64]]")  [return-value]

This looks like a legitimate error as np.mean(...) is already inferred as Any here. On master, passing that Any into the overloaded as_float(...) accidentally picks a precise scalar overload, which produces the return-value error. With this PR, the Any argument is treated as overload-ambiguous, so as_float(...) returns Any and the error disappears.
Colour may still have a real annotation issue, but the old error depended on over-precise overload selection from an Any input.

Your call @hauntsaninja