-
-
Notifications
You must be signed in to change notification settings - Fork 3k
Draft: Solve intermediate variable bug #19399
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Draft: Solve intermediate variable bug #19399
Conversation
lambda i: self.accept(args[i]), | ||
) | ||
|
||
# ??? QUESTION: Do we need to recompute arg_types and pass1_args here??? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems to work both with/without recomputing arg_types
here, but I am not sure.
# HACK: convert "Literal?" constraints to their non-literal versions. | ||
inner_constraints: list[Constraint] = [] | ||
for constraint in _inner_constraints: | ||
target = get_proper_type(constraint.target) | ||
inner_constraints.append( | ||
Constraint( | ||
constraint.original_type_var, | ||
constraint.op, | ||
( | ||
target.copy_modified(last_known_value=None) | ||
if isinstance(target, Instance) | ||
else target | ||
), | ||
) | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This hack is needed for testLiteralAndGenericWithUnion
and testLiteralMappingContext
.
The only purpose is to convert constraints like T ≷ Literal['foo"]?
to T ≷ str
, which, in these test cases, causes the joint solution to fail due to incompatible constraints like T <: Literal["foo"]
and T :> str
, hence the longer route of using the outer solution first is used.
Diff from mypy_primer, showing the effect of this PR on open source code: colour (https://github.com/colour-science/colour)
+ colour/utilities/array.py:2428: error: Argument 1 to "__call__" of "_FloatOp" has incompatible type "floating[_16Bit]"; expected "integer[_32Bit] | floating[_32Bit]" [arg-type]
+ colour/utilities/array.py:2428: error: Argument 1 to "__call__" of "_FloatOp" has incompatible type "floating[_32Bit]"; expected "integer[_16Bit] | floating[_16Bit]" [arg-type]
+ colour/utilities/array.py:2428: note: Both left and right operands are unions
+ colour/plotting/section.py:390: error: Argument 1 to "LineCollection" has incompatible type "ndarray[tuple[Any, ...], dtype[Any]]"; expected "Sequence[Buffer | _SupportsArray[dtype[Any]] | _NestedSequence[_SupportsArray[dtype[Any]]] | complex | bytes | str | _NestedSequence[complex | bytes | str]]" [arg-type]
pydantic (https://github.com/pydantic/pydantic)
- pydantic/aliases.py:29: error: Incompatible types in assignment (expression has type "list[str]", variable has type "list[int | str]") [assignment]
- pydantic/aliases.py:29: note: "list" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance
- pydantic/aliases.py:29: note: Consider using "Sequence" instead, which is covariant
- pydantic/aliases.py:29: error: Argument 1 to "list" has incompatible type "tuple[str | int, ...]"; expected "Iterable[str]" [arg-type]
scikit-learn (https://github.com/scikit-learn/scikit-learn)
- sklearn/linear_model/tests/test_ridge.py:744: error: Unsupported operand types for + ("list[None]" and "list[ABCMeta]") [operator]
operator (https://github.com/canonical/operator)
- ops/model.py:2937: error: Argument 2 to "_list_recursive" of "Container" has incompatible type "str | Path"; expected "Path" [arg-type]
- ops/model.py:3020: error: Argument 2 to "_list_recursive" of "Container" has incompatible type "str | Path"; expected "Path" [arg-type]
- ops/_private/harness.py:4000: error: Argument 3 to "_notice_matches" of "_TestingPebbleClient" has incompatible type "set[NoticeType | str] | None"; expected "set[str] | None" [arg-type]
pandas (https://github.com/pandas-dev/pandas)
+ pandas/core/arrays/datetimelike.py:2398: error: Unused "type: ignore" comment [unused-ignore]
+ pandas/core/arrays/sparse/accessor.py:443: error: Incompatible types in assignment (expression has type "ndarray[tuple[Any, ...], dtype[Any]]", variable has type "list[ndarray[tuple[int], dtype[Any]]]") [assignment]
+ pandas/core/arrays/sparse/accessor.py:444: error: Incompatible types in assignment (expression has type "ndarray[tuple[Any, ...], dtype[Any]]", variable has type "list[Any]") [assignment]
+ pandas/core/arrays/sparse/accessor.py:445: error: Incompatible types in assignment (expression has type "ndarray[tuple[Any, ...], dtype[Any]]", variable has type "list[Any]") [assignment]
+ pandas/core/indexes/interval.py:801: error: Incompatible types in assignment (expression has type "ndarray[tuple[Any, ...], dtype[Any]]", variable has type "list[ndarray[tuple[Any, ...], dtype[Any]] | ndarray[tuple[Any, ...], dtype[signedinteger[_32Bit | _64Bit]]]]") [assignment]
+ pandas/core/groupby/groupby.py:5650: error: Incompatible types in assignment (expression has type "ndarray[tuple[Any, ...], dtype[Any]]", variable has type "list[Any]") [assignment]
+ pandas/io/formats/csvs.py:141: error: List item 0 has incompatible type "Hashable"; expected "str" [list-item]
+ pandas/tests/series/accessors/test_dt_accessor.py:441: error: Unused "type: ignore" comment [unused-ignore]
+ pandas/tests/scalar/timestamp/test_timestamp.py:124: error: Unused "type: ignore" comment [unused-ignore]
+ pandas/tests/indexes/datetimes/test_scalar_compat.py:141: error: Unused "type: ignore" comment [unused-ignore]
+ pandas/tests/dtypes/test_missing.py:806: error: Unused "type: ignore" comment [unused-ignore]
+ pandas/tests/dtypes/test_missing.py:828: error: Unused "type: ignore" comment [unused-ignore]
+ pandas/conftest.py:1664: error: Unused "type: ignore" comment [unused-ignore]
prefect (https://github.com/PrefectHQ/prefect)
- src/prefect/utilities/callables.py:579: error: Incompatible types in assignment (expression has type "list[None]", variable has type "list[Optional[expr]]") [assignment]
- src/prefect/utilities/callables.py:579: note: "list" is invariant -- see https://mypy.readthedocs.io/en/stable/common_issues.html#variance
- src/prefect/utilities/callables.py:579: note: Consider using "Sequence" instead, which is covariant
- src/prefect/utilities/callables.py:581: error: Unsupported operand types for + ("list[None]" and "list[Optional[expr]]") [operator]
+ src/prefect/client/orchestration/__init__.py:1088: error: Function does not return a value (it only ever returns None) [func-returns-value]
artigraph (https://github.com/artigraph/artigraph)
+ src/arti/graphs/__init__.py:181: error: Unused "type: ignore" comment [unused-ignore]
Expression (https://github.com/cognitedata/Expression)
+ expression/collections/block.py:271: error: Incompatible return value type (got "int", expected "_TSourceSum | Literal[0]") [return-value]
+ expression/collections/block.py:271: error: Argument 1 to "sum" has incompatible type "tuple[_TSourceSum | Literal[0], ...]"; expected "Iterable[bool]" [arg-type]
+ expression/collections/block.py:934: error: Incompatible return value type (got "int", expected "_TSourceSum | Literal[0]") [return-value]
+ expression/collections/block.py:934: error: Argument 1 to "sum" has incompatible type "Block[_TSourceSum | Literal[0]]"; expected "Iterable[bool]" [arg-type]
jax (https://github.com/google/jax)
+ jax/_src/numpy/lax_numpy.py:2224: error: Unsupported operand types for - ("Sequence[int | Any]" and "int") [operator]
+ jax/_src/numpy/lax_numpy.py:2224: note: Left operand is of type "Sequence[int | Any] | int | Any"
hydra-zen (https://github.com/mit-ll-responsible-ai/hydra-zen)
- src/hydra_zen/wrapper/_implementations.py:1756: error: Unsupported operand types for + ("list[None]" and "list[str]") [operator]
altair (https://github.com/vega/altair)
+ tools/generate_schema_wrapper.py:488: error: Dict entry 0 has incompatible type "str": "OverridesItem[MethodSchemaGenerator]"; expected "str": "OverridesItem[SchemaGenerator]" [dict-item]
static-frame (https://github.com/static-frame/static-frame)
+ static_frame/core/store.py:181: error: Argument 1 to "extend" of "list" has incompatible type "Sequence[TLabel]"; expected "Iterable[str]" [arg-type]
+ static_frame/core/store.py:181: error: Argument 1 to "extend" of "list" has incompatible type "Sequence[TLabel]"; expected "Iterable[tuple[str, ...]]" [arg-type]
+ static_frame/core/store.py:183: error: Argument 1 to "extend" of "list" has incompatible type "range"; expected "Iterable[str]" [arg-type]
+ static_frame/core/store.py:183: note: Following member(s) of "range" have conflicts:
+ static_frame/core/store.py:183: note: Expected:
+ static_frame/core/store.py:183: note: def __iter__(self) -> Iterator[str]
+ static_frame/core/store.py:183: note: Got:
+ static_frame/core/store.py:183: note: def __iter__(self) -> Iterator[int]
+ static_frame/core/store.py:183: error: Argument 1 to "extend" of "list" has incompatible type "range"; expected "Iterable[tuple[str, ...]]" [arg-type]
+ static_frame/core/store.py:183: note: Following member(s) of "range" have conflicts:
+ static_frame/core/store.py:183: note: Expected:
+ static_frame/core/store.py:183: note: def __iter__(self) -> Iterator[tuple[str, ...]]
+ static_frame/core/store.py:183: note: Got:
+ static_frame/core/store.py:183: note: def __iter__(self) -> Iterator[int]
steam.py (https://github.com/Gobot1234/steam.py)
+ steam/abc.py:552: error: Value of type variable "AppT" of "FavouriteBadge" cannot be "PartialApp[str | None] | tuple[str, int]" [type-var]
- steam/state.py:1453: error: Argument 2 to "pop" of "dict" has incompatible type "None"; expected "ClanInvite | GroupInvite" [arg-type]
+ steam/state.py:1453: error: Incompatible types in assignment (expression has type "ClanInvite | GroupInvite | None", variable has type "ClanInvite | GroupInvite") [assignment]
core (https://github.com/home-assistant/core)
+ homeassistant/helpers/service.py:982: error: Argument 1 to "HassJob" has incompatible type "Callable[[ServiceCall], Coroutine[Any, Any, ServiceResponse | EntityServiceResponse] | ServiceResponse | EntityServiceResponse | None]"; expected "Callable[[VarArg([ServiceCall] | [ServiceCall]), KwArg([ServiceCall] | [ServiceCall])], Coroutine[Any, Any, ServiceResponse | EntityServiceResponse] | JsonObjectType | EntityServiceResponse | JsonObjectType | EntityServiceResponse | None]" [arg-type]
+ homeassistant/components/bthome/coordinator.py:48: error: Returning Any from function declared to return "bool" [no-any-return]
+ homeassistant/components/xiaomi_ble/coordinator.py:70: error: Returning Any from function declared to return "bool" [no-any-return]
+ homeassistant/components/motioneye/media_source.py:96: error: Unused "type: ignore" comment [unused-ignore]
scipy-stubs (https://github.com/scipy/scipy-stubs)
+ tests/sparse/test_construct.pyi:125: error: Expression is of type "coo_array[_ScalarT, tuple[int, int]]", not "coo_array[float64, tuple[int, int]]" [assert-type]
+ tests/sparse/test_construct.pyi:126: error: Expression is of type "csc_array[_ScalarT]", not "csc_array[float64]" [assert-type]
+ tests/sparse/test_construct.pyi:127: error: Expression is of type "csr_array[_ScalarT, tuple[int, int]]", not "csr_array[float64, tuple[int, int]]" [assert-type]
+ tests/sparse/test_construct.pyi:128: error: Expression is of type "coo_array[_ScalarT, tuple[int, int]]", not "coo_array[complex128, tuple[int, int]]" [assert-type]
+ tests/sparse/test_construct.pyi:129: error: Expression is of type "coo_array[_ScalarT, tuple[int, int]]", not "coo_array[complex128, tuple[int, int]]" [assert-type]
+ tests/sparse/test_construct.pyi:130: error: Expression is of type "coo_array[_ScalarT, tuple[int, int]]", not "coo_array[complex128, tuple[int, int]]" [assert-type]
spark (https://github.com/apache/spark)
+ python/pyspark/ml/util.py:1166: error: Unused "type: ignore" comment [unused-ignore]
+ python/pyspark/ml/tuning.py:257: error: Value of type variable "JP" of "_from_java" of "JavaParams" cannot be "Estimator[Any]" [type-var]
+ python/pyspark/ml/tuning.py:440: error: Value of type variable "RL" of "loadParamsInstance" of "DefaultParamsReader" cannot be "Estimator[Any]" [type-var]
+ python/pyspark/ml/classification.py:3662: error: Value of type variable "JP" of "_from_java" of "JavaParams" cannot be "Classifier[Any]" [type-var]
+ python/pyspark/ml/classification.py:3920: error: Value of type variable "JP" of "_from_java" of "JavaParams" cannot be "Classifier[Any]" [type-var]
+ python/pyspark/ml/classification.py:3976: error: Unused "type: ignore" comment [unused-ignore]
antidote (https://github.com/Finistere/antidote)
+ tests/lib/interface/test_custom.py:434: error: Unused "type: ignore" comment [unused-ignore]
+ tests/lib/interface/test_custom.py:437: error: Unused "type: ignore" comment [unused-ignore]
+ tests/lib/interface/test_custom.py:440: error: Unused "type: ignore" comment [unused-ignore]
+ tests/lib/interface/test_custom.py:443: error: Unused "type: ignore" comment [unused-ignore]
+ tests/lib/interface/test_custom.py:446: error: Unused "type: ignore" comment [unused-ignore]
+ tests/lib/interface/test_custom.py:449: error: Unused "type: ignore" comment [unused-ignore]
strawberry (https://github.com/strawberry-graphql/strawberry)
+ strawberry/experimental/pydantic/error_type.py:118: error: Value of type variable "T" of "_wrap_dataclass" cannot be "WithStrawberryObjectDefinition" [type-var]
discord.py (https://github.com/Rapptz/discord.py)
- discord/ext/commands/bot.py:1316: error: Argument 1 to "find" has incompatible type "Callable[[str], bool]"; expected "Callable[[list[str] | str], Any]" [arg-type]
schemathesis (https://github.com/schemathesis/schemathesis)
+ src/schemathesis/specs/openapi/_hypothesis.py:397: error: Incompatible types in assignment (expression has type "SearchStrategy[dict[str, Any]]", variable has type "SearchStrategy[None]") [assignment]
- src/schemathesis/specs/openapi/_hypothesis.py:397: error: Argument 1 to "map" of "SearchStrategy" has incompatible type "Callable[[dict[str, Any]], dict[str, Any]]"; expected "Callable[[dict[str, Any]], None]" [arg-type]
+ src/schemathesis/specs/openapi/_hypothesis.py:399: error: Incompatible types in assignment (expression has type "SearchStrategy[dict[str, Any]]", variable has type "SearchStrategy[None]") [assignment]
- src/schemathesis/specs/openapi/_hypothesis.py:399: error: Argument 1 to "map" of "SearchStrategy" has incompatible type "Callable[[dict[str, Any]], dict[str, Any]]"; expected "Callable[[None], None]" [arg-type]
+ src/schemathesis/specs/openapi/_hypothesis.py:399: error: Argument 1 to "map" of "SearchStrategy" has incompatible type "Callable[[dict[str, Any]], dict[str, Any]]"; expected "Callable[[None], dict[str, Any]]" [arg-type]
xarray (https://github.com/pydata/xarray)
+ xarray/core/dataset.py: note: In member "drop_dims" of class "Dataset":
+ xarray/core/dataset.py:6108: error: Argument 1 to "set" has incompatible type "Frozen[Hashable, int]"; expected "Iterable[str | None]" [arg-type]
+ xarray/core/dataset.py:6108: note: Left operand is of type "set[str] | set[Hashable]"
+ xarray/core/dataset.py: note: At top level:
+ xarray/core/dataarray.py: note: In function "_infer_coords_and_dims":
+ xarray/core/dataarray.py:163: error: No overload variant of "__setitem__" of "list" matches argument types "int", "Hashable" [call-overload]
+ xarray/core/dataarray.py:163: note: Possible overload variants:
+ xarray/core/dataarray.py:163: note: def __setitem__(self, SupportsIndex, str, /) -> None
+ xarray/core/dataarray.py:163: note: def __setitem__(self, slice[Any, Any, Any], Iterable[str], /) -> None
+ xarray/core/dataarray.py:186: error: Incompatible types in assignment (expression has type "Hashable", variable has type "str") [assignment]
+ xarray/core/dataarray.py: note: At top level:
+ xarray/tests/test_dataarray.py:423: error: Unused "type: ignore" comment [unused-ignore]
zulip (https://github.com/zulip/zulip)
+ zproject/config.py:38: error: Argument "fallback" to "get" of "RawConfigParser" has incompatible type "str | None"; expected "None" [arg-type]
sympy (https://github.com/sympy/sympy)
+ sympy/core/add.py:384: error: Unused "type: ignore" comment [unused-ignore]
|
This still isn't quite correct; there are persisting issues with considering the outer context, for examplefrom typing import TypedDict
class A: ...
class B(A): ...
class OverridesItem[T](TypedDict):
tp: type[T]
d: dict[str, OverridesItem[A]] = {"foo": OverridesItem(tp=B)} gives
with the current version of the PR. It seems that these issues stem from "solving" variables too eagerly. In the example above, |
Looking for feedback, as this was mostly a trial-and-error process.
self.infer_function_type_arguments_using_context
. Instead, we now use a functioninfer_constraints_from_context
that only yields the constraints.infer_function_type_arguments
to now include the context.We compute 2 sets of constraints: the "outer constraints", determined by the context, and the "inner constraints" determined by the callable type. From this, we compute 2 solutions:
the return type of the outer solution is a union, and any member of that union is a subtype of the joint return type.1New Tests
testBinaryOperatorContext
mypy-playgroundtestContextFreeConcatInvariantType
mypy-playgroundtestInContextConcatInvariantType
mypy-playgroundtestLiteralMappingContext
: mypy-playground new test based on a failure in pandas-stubs.Modified Tests
testRecursiveAliasWithRecursiveInstance
: improved inference2testCallerTupleVarArgsAndGenericCalleeVarArg
now revealsint | None
rather thanLiteral[1]? | None
3testInferenceAgainstGenericCallableGenericProtocol
now revealsF[T | None]
rather thandef [T] (T|None) -> T|None
4testInferenceAgainstGenericParamSpecPopOn
now revealsdef (U) -> U
rather thandef [T] (T) -> T
5Footnotes
This was needed to fix
testLiteralAndGenericWithUnion
, because the outer solution wasint | Literal["foo"]
whereas the joint solution wasLiteral["foo"]?
which got converted intostr
later. ↩previously,
a=b
followed bya=[[b]]
madea
be inferred as__main__.B
followed bybuiltins.list[Union[__main__.A, typing.Sequence[Union[__main__.A, ...]]]]
. Now, it gets more precisely inferred asbuiltins.list[builtins.list[__main__.B]]
. And afterwards,join(a,b)
get more precisely inferred astyping.Sequence[typing.Sequence[__main__.B]]
rather thantyping.Sequence[Union[__main__.A, typing.Sequence[Union[__main__.A, ...]]]]
previously. See: https://mypy-play.net/?mypy=latest&python=3.12&gist=619a7fa6c3ec9d75bd1492bf8e68432c. The new result is also more in line with the inference by pyright ↩same as pyright ↩
same as pyright ↩
same as pyright ↩