`functools`: Add typing for decorated function arguments in cache API [take 2] #10523

jakkdl · 2023-07-31T16:01:25Z

Giving #9768 another attempt, adding WMOkiishis suggested changes to see if that helps with mypy primer output.

Also rebased on top of main/

…functools.lru_cache()

primer. This reverts commit 89e9326.

github-actions · 2023-07-31T16:24:03Z

Diff from mypy_primer, showing the effect of this PR on open source code:

mitmproxy (https://github.com/mitmproxy/mitmproxy)
+ mitmproxy/coretypes/serializable.py:82: error: Too few arguments  [call-arg]
+ mitmproxy/coretypes/serializable.py:90: error: Missing positional argument "cls" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ mitmproxy/coretypes/serializable.py:98: error: Too few arguments  [call-arg]
+ mitmproxy/proxy/mode_specs.py:164: error: Missing positional argument "spec" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ mitmproxy/connection.py:181: error: Missing positional argument "spec" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ mitmproxy/connection.py:181: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "str"; expected "type[<nothing>]"  [arg-type]
+ mitmproxy/utils/asyncio_utils.py:56: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "Any | str"; expected "tuple[Any, ...] | None"  [arg-type]
+ mitmproxy/tools/console/common.py:811: error: Argument "error_message" to "__call__" of "_lru_cache_wrapper" has incompatible type "str | None"; expected "str"  [arg-type]
+ mitmproxy/tools/console/common.py:848: error: Incompatible types in assignment (expression has type "_lru_cache_wrapper[[NamedArg(RenderMode, 'render_mode'), NamedArg(bool, 'focused'), NamedArg(str, 'marked'), NamedArg(str | None, 'is_replay'), NamedArg(str, 'request_method'), NamedArg(str, 'request_scheme'), NamedArg(str, 'request_host'), NamedArg(str, 'request_path'), NamedArg(str, 'request_url'), NamedArg(str, 'request_http_version'), NamedArg(float, 'request_timestamp'), NamedArg(bool, 'request_is_push_promise'), NamedArg(bool, 'intercepted'), NamedArg(int | None, 'response_code'), NamedArg(str | None, 'response_reason'), NamedArg(int | None, 'response_content_length'), NamedArg(str | None, 'response_content_type'), NamedArg(float | None, 'duration'), NamedArg(str | None, 'error_message')], Any]", variable has type "_lru_cache_wrapper[[NamedArg(RenderMode, 'render_mode'), NamedArg(bool, 'focused'), NamedArg(str, 'marked'), NamedArg(bool, 'is_replay'), NamedArg(str, 'request_method'), NamedArg(str, 'request_scheme'), NamedArg(str, 'request_host'), NamedArg(str, 'request_path'), NamedArg(str, 'request_url'), NamedArg(str, 'request_http_version'), NamedArg(float, 'request_timestamp'), NamedArg(bool, 'request_is_push_promise'), NamedArg(bool, 'intercepted'), NamedArg(int | None, 'response_code'), NamedArg(str | None, 'response_reason'), NamedArg(int | None, 'response_content_length'), NamedArg(str | None, 'response_content_type'), NamedArg(float | None, 'duration'), NamedArg(str | None, 'error_message')], Any]")  [assignment]
+ mitmproxy/tools/console/common.py:853: error: Argument "is_replay" to "__call__" of "_lru_cache_wrapper" has incompatible type "str | None"; expected "bool"  [arg-type]
+ mitmproxy/command.py:285: error: Too few arguments  [call-arg]
+ mitmproxy/command.py:285: error: Argument 1 has incompatible type "str"; expected "CommandManager"  [arg-type]
+ mitmproxy/test/tflow.py:134: error: Missing positional argument "spec" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ mitmproxy/test/tflow.py:134: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "str"; expected "type[<nothing>]"  [arg-type]
+ mitmproxy/test/tflow.py:235: error: Missing positional argument "spec" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ mitmproxy/test/tflow.py:235: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "str"; expected "type[<nothing>]"  [arg-type]
+ mitmproxy/master.py:115: error: Missing positional argument "spec" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ mitmproxy/tools/console/commander/commander.py:77: error: Too few arguments  [call-arg]
+ mitmproxy/tools/console/commander/commander.py:77: error: Argument 1 has incompatible type "str"; expected "CommandManager"  [arg-type]
+ mitmproxy/tools/console/commander/commander.py:109: error: Too few arguments  [call-arg]
+ mitmproxy/tools/console/commander/commander.py:110: error: Argument 1 has incompatible type "str"; expected "CommandManager"  [arg-type]
+ mitmproxy/proxy/server.py:556: error: Missing positional argument "spec" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ mitmproxy/proxy/server.py:556: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "str"; expected "type[<nothing>]"  [arg-type]
+ mitmproxy/proxy/mode_servers.py:113: error: Missing positional argument "spec" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ mitmproxy/proxy/mode_servers.py:113: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "str"; expected "type[<nothing>]"  [arg-type]
+ mitmproxy/addons/clientplayback.py:97: error: Missing positional argument "spec" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ mitmproxy/addons/proxyserver.py:102: error: Missing positional argument "spec" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ mitmproxy/addons/proxyserver.py:102: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "str"; expected "type[<nothing>]"  [arg-type]
+ mitmproxy/addons/proxyserver.py:245: error: Missing positional argument "spec" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ mitmproxy/addons/proxyserver.py:282: error: Missing positional argument "spec" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ mitmproxy/tools/console/keymap.py:87: error: "Callable[[Keymap, Keymap, str], Binding | None]" has no attribute "cache_clear"  [attr-defined]
+ mitmproxy/tools/console/flowview.py:68: error: "Callable[[FlowDetails, FlowDetails, Any, Any, Any], Any]" has no attribute "cache_clear"  [attr-defined]
+ mitmproxy/tools/console/flowview.py:303: error: Too few arguments  [call-arg]
+ mitmproxy/tools/console/flowview.py:304: error: Argument 2 has incompatible type "int"; expected "FlowDetails"  [arg-type]
+ mitmproxy/tools/console/flowlist.py:59: error: "Callable[[FlowListWalker, FlowListWalker, int], tuple[FlowItem | None, int | None]]" has no attribute "cache_clear"  [attr-defined]
+ mitmproxy/tools/console/flowlist.py:78: error: Too few arguments  [call-arg]
+ mitmproxy/tools/console/flowlist.py:81: error: Too few arguments  [call-arg]
+ mitmproxy/tools/console/quickhelp.py:175: error: Too few arguments  [call-arg]
+ mitmproxy/tools/console/quickhelp.py:175: error: Argument 1 has incompatible type "str"; expected "Keymap"  [arg-type]

pip (https://github.com/pypa/pip)
+ src/pip/_internal/metadata/base.py:406: error: Too few arguments  [call-arg]
+ src/pip/_internal/models/link.py:245: error: Missing positional argument "url" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ src/pip/_internal/models/link.py:245: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "str"; expected "type[LinkHash]"  [arg-type]
+ src/pip/_internal/req/req_install.py:528: error: Too few arguments  [call-arg]
+ src/pip/_internal/req/req_install.py:554: error: Too few arguments  [call-arg]
+ src/pip/_internal/index/package_finder.py:890: error: Too few arguments  [call-arg]
+ src/pip/_internal/index/package_finder.py:890: error: Argument 1 has incompatible type "str"; expected "PackageFinder"  [arg-type]
+ src/pip/_internal/index/package_finder.py:908: error: Too few arguments  [call-arg]
+ src/pip/_internal/index/package_finder.py:909: error: Argument 1 has incompatible type "str | None"; expected "PackageFinder"  [arg-type]
+ src/pip/_internal/self_outdated_check.py:177: error: Too few arguments  [call-arg]
+ src/pip/_internal/self_outdated_check.py:177: error: Argument 1 has incompatible type "str"; expected "PackageFinder"  [arg-type]
+ src/pip/_internal/wheel_builder.py:73: error: Too few arguments  [call-arg]
+ src/pip/_internal/distributions/sdist.py:114: error: Too few arguments  [call-arg]
+ src/pip/_internal/resolution/resolvelib/factory.py:279: error: Too few arguments  [call-arg]
+ src/pip/_internal/resolution/resolvelib/factory.py:604: error: Too few arguments  [call-arg]
+ src/pip/_internal/resolution/resolvelib/factory.py:604: error: Argument 1 has incompatible type "NormalizedName"; expected "PackageFinder"  [arg-type]
+ src/pip/_internal/commands/list.py:239: error: Too few arguments  [call-arg]
+ src/pip/_internal/commands/list.py:239: error: Argument 1 has incompatible type "NormalizedName"; expected "PackageFinder"  [arg-type]
+ src/pip/_internal/commands/index.py:119: error: Too few arguments  [call-arg]

manticore (https://github.com/trailofbits/manticore)
+ manticore/core/smtlib/solver.py:505: error: Signature of "can_be_true" incompatible with supertype "Solver"  [override]
+ manticore/core/smtlib/solver.py:505: note:      Superclass:
+ manticore/core/smtlib/solver.py:505: note:          def can_be_true(self, constraints: Any, expression: Any = ...) -> bool
+ manticore/core/smtlib/solver.py:505: note:      Subclass:
+ manticore/core/smtlib/solver.py:505: note:          _lru_cache_wrapper[[SMTLIBSolver, ConstraintSet, Union[bool, Bool]], bool]
+ manticore/core/smtlib/solver.py:605: error: Signature of "get_all_values" incompatible with supertype "Solver"  [override]
+ manticore/core/smtlib/solver.py:605: note:      Superclass:
+ manticore/core/smtlib/solver.py:605: note:          def get_all_values(self, constraints: Any, x: Any, maxcnt: Any = ..., silent: Any = ...) -> Any
+ manticore/core/smtlib/solver.py:605: note:      Subclass:
+ manticore/core/smtlib/solver.py:605: note:          _lru_cache_wrapper[[SMTLIBSolver, ConstraintSet, Any, Optional[int], bool], Any]

pandas (https://github.com/pandas-dev/pandas)
+ pandas/core/dtypes/cast.py:589: error: Unused "type: ignore" comment  [unused-ignore]
+ pandas/core/groupby/ops.py:395: error: Too few arguments  [call-arg]
+ pandas/core/groupby/ops.py:395: error: Argument 1 has incompatible type "str"; expected "WrappedCythonOp"  [arg-type]
+ pandas/core/groupby/ops.py:395: error: Argument 2 has incompatible type "str"; expected "type[WrappedCythonOp]"  [arg-type]
+ pandas/core/groupby/ops.py:395: error: Argument 4 has incompatible type "bool"; expected "str"  [arg-type]

aiohttp (https://github.com/aio-libs/aiohttp)
+ aiohttp/connector.py:932:24: error: Too few arguments  [call-arg]
+ aiohttp/connector.py:932:47: error: Argument 1 has incompatible type "bool"; expected "TCPConnector"  [arg-type]
+ aiohttp/connector.py:938:24: error: Too few arguments  [call-arg]
+ aiohttp/connector.py:938:47: error: Argument 1 has incompatible type "bool"; expected "TCPConnector"  [arg-type]
+ aiohttp/connector.py:939:20: error: Too few arguments  [call-arg]
+ aiohttp/connector.py:939:43: error: Argument 1 has incompatible type "bool"; expected "TCPConnector"  [arg-type]

rich (https://github.com/Textualize/rich)
+ rich/theme.py:25: error: Missing positional argument "style_definition" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ rich/theme.py:25: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "str"; expected "type[Style]"  [arg-type]
+ rich/theme.py:54: error: Missing positional argument "style_definition" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ rich/theme.py:54: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "str"; expected "type[Style]"  [arg-type]
+ rich/style.py:147: error: Missing positional argument "color" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ rich/style.py:147: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "str"; expected "type[Color]"  [arg-type]
+ rich/style.py:368: error: Too few arguments  [call-arg]
+ rich/style.py:368: error: Argument 1 has incompatible type "ColorSystem"; expected "Color"  [arg-type]
+ rich/style.py:371: error: Too few arguments  [call-arg]
+ rich/style.py:371: error: Argument 1 has incompatible type "ColorSystem"; expected "Color"  [arg-type]
+ rich/style.py:391: error: Missing positional argument "style_definition" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ rich/style.py:391: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "str"; expected "type[Style]"  [arg-type]
+ rich/style.py:527: error: Missing positional argument "color" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ rich/style.py:527: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "str"; expected "type[Color]"  [arg-type]
+ rich/style.py:554: error: Missing positional argument "color" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ rich/style.py:554: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "str"; expected "type[Color]"  [arg-type]
+ rich/style.py:758: error: Too few arguments  [call-arg]
+ rich/style.py:758: error: Argument 1 has incompatible type "Style | None"; expected "Style"  [arg-type]
+ rich/segment.py:169: error: Too few arguments  [call-arg]
+ rich/segment.py:169: error: Argument 2 has incompatible type "int"; expected "type[Segment]"  [arg-type]
+ rich/markup.py:160: error: Missing positional argument "style" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ rich/markup.py:160: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "str"; expected "type[Style]"  [arg-type]
+ rich/markup.py:217: error: Missing positional argument "style" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ rich/markup.py:217: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "str"; expected "type[Style]"  [arg-type]
+ rich/jupyter.py:72: error: Too few arguments  [call-arg]
+ rich/jupyter.py:72: error: Argument 1 has incompatible type "TerminalTheme"; expected "Style"  [arg-type]
+ rich/console.py:1476: error: Missing positional argument "style_definition" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ rich/console.py:1476: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "str"; expected "type[Style]"  [arg-type]
+ rich/console.py:2210: error: Too few arguments  [call-arg]
+ rich/console.py:2210: error: Argument 1 has incompatible type "TerminalTheme"; expected "Style"  [arg-type]
+ rich/console.py:2222: error: Too few arguments  [call-arg]
+ rich/console.py:2222: error: Argument 1 has incompatible type "TerminalTheme"; expected "Style"  [arg-type]
+ rich/color.py:552: error: Too few arguments  [call-arg]
+ rich/color.py:552: error: Argument 1 has incompatible type "ColorTriplet"; expected "Palette"  [arg-type]
+ rich/color.py:565: error: Too few arguments  [call-arg]
+ rich/color.py:565: error: Argument 1 has incompatible type "ColorTriplet"; expected "Palette"  [arg-type]
+ rich/ansi.py:175: error: Missing positional argument "style_definition" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ rich/ansi.py:175: error: Argument 1 to "__call__" of "_lru_cache_wrapper" has incompatible type "str"; expected "type[Style]"  [arg-type]
+ rich/progress_bar.py:144: error: Too few arguments  [call-arg]
+ rich/progress_bar.py:145: error: Argument 1 has incompatible type "Style"; expected "ProgressBar"  [arg-type]
+ rich/progress_bar.py:145: error: Argument 2 has incompatible type "Style"; expected "ProgressBar"  [arg-type]
+ rich/progress_bar.py:145: error: Argument 3 has incompatible type "str | None"; expected "Style"  [arg-type]
+ rich/progress_bar.py:145: error: Argument 4 has incompatible type "bool"; expected "Style"  [arg-type]

dacite (https://github.com/konradhalas/dacite)
+ dacite/core.py:51: error: Argument "localns" to "get_type_hints" has incompatible type "FrozenDict | None"; expected "dict[str, Any] | None"  [arg-type]

pylint (https://github.com/pycqa/pylint)
+ pylint/checkers/utils.py:2238: error: "_lru_cache_wrapper" expects 2 type arguments, but 1 given  [type-arg]
+ pylint/message/message_definition_store.py:79: error: Too few arguments  [call-arg]
+ pylint/message/message_definition_store.py:79: error: Argument 1 has incompatible type "str"; expected "MessageDefinitionStore"  [arg-type]
+ pylint/message/message_definition_store.py:88: error: Too few arguments  [call-arg]
+ pylint/message/message_definition_store.py:89: error: Argument 1 has incompatible type "str"; expected "MessageDefinitionStore"  [arg-type]
+ pylint/lint/message_state_handler.py:123: error: Too few arguments  [call-arg]
+ pylint/lint/message_state_handler.py:123: error: Argument 1 has incompatible type "str"; expected "MessageDefinitionStore"  [arg-type]
+ pylint/lint/message_state_handler.py:151: error: Too few arguments  [call-arg]
+ pylint/lint/message_state_handler.py:151: error: Argument 1 has incompatible type "str"; expected "MessageDefinitionStore"  [arg-type]
+ pylint/lint/pylinter.py:1261: error: Too few arguments  [call-arg]
+ pylint/lint/pylinter.py:1261: error: Argument 1 has incompatible type "str"; expected "MessageDefinitionStore"  [arg-type]
+ pylint/lint/pylinter.py:1288: error: Too few arguments  [call-arg]
+ pylint/lint/pylinter.py:1288: error: Argument 1 has incompatible type "str"; expected "MessageDefinitionStore"  [arg-type]
+ pylint/checkers/format.py:446: error: Too few arguments  [call-arg]
+ pylint/checkers/format.py:447: error: Argument 1 has incompatible type "str"; expected "MessageDefinitionStore"  [arg-type]

mypy (https://github.com/python/mypy)
+ mypy/find_sources.py:160: error: Too few arguments  [call-arg]
+ mypy/find_sources.py:160: error: Argument 1 has incompatible type "str"; expected "SourceFinder"  [arg-type]
+ mypy/find_sources.py:202: error: Too few arguments  [call-arg]
+ mypy/find_sources.py:202: error: Argument 1 has incompatible type "str"; expected "SourceFinder"  [arg-type]

isort (https://github.com/pycqa/isort)
+ isort/deprecated/finders.py:296: error: Too few arguments  [call-arg]
+ isort/deprecated/finders.py:296: error: Argument 1 has incompatible type "str"; expected "RequirementsFinder"  [arg-type]
+ isort/deprecated/finders.py:327: error: Too few arguments  [call-arg]
+ isort/deprecated/finders.py:327: error: Argument 1 has incompatible type "str"; expected "RequirementsFinder"  [arg-type]

psycopg (https://github.com/psycopg/psycopg)
+ psycopg/psycopg/rows.py:145: error: Argument 2 to "__call__" of "_lru_cache_wrapper" has incompatible type "*Generator[bytes | None, None, None]"; expected "bytes"  [arg-type]

ibis (https://github.com/ibis-project/ibis)
+ ibis/backends/datafusion/__init__.py:334: error: Missing positional argument "cls" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ ibis/backends/polars/__init__.py:347: error: Missing positional argument "cls" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ ibis/backends/pandas/__init__.py:228: error: Missing positional argument "cls" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]
+ ibis/backends/base/sql/__init__.py:360: error: Missing positional argument "cls" in call to "__call__" of "_lru_cache_wrapper"  [call-arg]

jax (https://github.com/google/jax)
+ jax/_src/sharding_impls.py:74: error: Signature of "devices_indices_map" incompatible with supertype "Sharding"  [override]
+ jax/_src/sharding_impls.py:74: note:      Superclass:
+ jax/_src/sharding_impls.py:74: note:          def devices_indices_map(self, global_shape: tuple[int, ...]) -> Mapping[Device?, tuple[slice, ...] | None]
+ jax/_src/sharding_impls.py:74: note:      Subclass:
+ jax/_src/sharding_impls.py:74: note:          _lru_cache_wrapper[[XLACompatibleSharding, tuple[int, ...]], Mapping[Device?, tuple[slice, ...]]]
+ jax/_src/sharding_impls.py:85: error: Signature of "shard_shape" incompatible with supertype "Sharding"  [override]
+ jax/_src/sharding_impls.py:85: note:      Superclass:
+ jax/_src/sharding_impls.py:85: note:          def shard_shape(self, global_shape: tuple[int, ...]) -> tuple[int, ...]
+ jax/_src/sharding_impls.py:85: note:      Subclass:
+ jax/_src/sharding_impls.py:85: note:          _lru_cache_wrapper[[XLACompatibleSharding, tuple[int, ...]], tuple[int, ...]]
+ jax/_src/sharding_impls.py:344: error: Signature of "_to_xla_hlo_sharding" incompatible with supertype "XLACompatibleSharding"  [override]
+ jax/_src/sharding_impls.py:344: note:      Superclass:
+ jax/_src/sharding_impls.py:344: note:          def _to_xla_hlo_sharding(self, num_dimensions: int) -> Any
+ jax/_src/sharding_impls.py:344: note:      Subclass:
+ jax/_src/sharding_impls.py:344: note:          _lru_cache_wrapper[[NamedSharding, int, SPMDAxisContext | ShardingContext | None], Any]
+ jax/_src/sharding_impls.py:521: error: Signature of "devices_indices_map" incompatible with supertype "XLACompatibleSharding"  [override]
+ jax/_src/sharding_impls.py:521: note:      Superclass:
+ jax/_src/sharding_impls.py:521: note:          _lru_cache_wrapper[[XLACompatibleSharding, tuple[int, ...]], Mapping[Device?, tuple[slice, ...]]]
+ jax/_src/sharding_impls.py:521: note:      Subclass:
+ jax/_src/sharding_impls.py:521: note:          _lru_cache_wrapper[[PmapSharding, tuple[int, ...]], Mapping[Device?, tuple[slice, ...]]]
+ jax/_src/sharding_impls.py:521: error: Signature of "devices_indices_map" incompatible with supertype "Sharding"  [override]
+ jax/_src/sharding_impls.py:521: note:      Superclass:
+ jax/_src/sharding_impls.py:521: note:          def devices_indices_map(self, global_shape: tuple[int, ...]) -> Mapping[Device?, tuple[slice, ...] | None]
+ jax/_src/sharding_impls.py:521: note:      Subclass:
+ jax/_src/sharding_impls.py:521: note:          _lru_cache_wrapper[[PmapSharding, tuple[int, ...]], Mapping[Device?, tuple[slice, ...]]]
+ jax/_src/sharding_impls.py:522: error: Too few arguments  [call-arg]
+ jax/_src/sharding_impls.py:522: error: Argument 1 has incompatible type "tuple[int, ...]"; expected "PmapSharding"  [arg-type]
+ jax/_src/sharding_impls.py:545: error: Signature of "shard_shape" incompatible with supertype "XLACompatibleSharding"  [override]
+ jax/_src/sharding_impls.py:545: note:      Superclass:
+ jax/_src/sharding_impls.py:545: note:          _lru_cache_wrapper[[XLACompatibleSharding, tuple[int, ...]], tuple[int, ...]]
+ jax/_src/sharding_impls.py:545: note:      Subclass:
+ jax/_src/sharding_impls.py:545: note:          _lru_cache_wrapper[[PmapSharding, tuple[int, ...]], tuple[int, ...]]
+ jax/_src/sharding_impls.py:545: error: Signature of "shard_shape" incompatible with supertype "Sharding"  [override]
+ jax/_src/sharding_impls.py:545: note:      Superclass:
+ jax/_src/sharding_impls.py:545: note:          def shard_shape(self, global_shape: tuple[int, ...]) -> tuple[int, ...]
+ jax/_src/sharding_impls.py:545: note:      Subclass:
+ jax/_src/sharding_impls.py:545: note:          _lru_cache_wrapper[[PmapSharding, tuple[int, ...]], tuple[int, ...]]
+ jax/_src/sharding_impls.py:690: error: Signature of "_to_xla_hlo_sharding" incompatible with supertype "XLACompatibleSharding"  [override]
+ jax/_src/sharding_impls.py:690: note:      Superclass:
+ jax/_src/sharding_impls.py:690: note:          def _to_xla_hlo_sharding(self, num_dimensions: int) -> Any
+ jax/_src/sharding_impls.py:690: note:      Subclass:
+ jax/_src/sharding_impls.py:690: note:          _lru_cache_wrapper[[PositionalSharding, int], Any]
+ jax/_src/sharding_impls.py:826: error: Signature of "devices_indices_map" incompatible with supertype "XLACompatibleSharding"  [override]
+ jax/_src/sharding_impls.py:826: note:      Superclass:
+ jax/_src/sharding_impls.py:826: note:          _lru_cache_wrapper[[XLACompatibleSharding, tuple[int, ...]], Mapping[Device?, tuple[slice, ...]]]
+ jax/_src/sharding_impls.py:826: note:      Subclass:
+ jax/_src/sharding_impls.py:826: note:          _lru_cache_wrapper[[GSPMDSharding, tuple[int, ...]], Mapping[Device?, tuple[slice, ...]]]
+ jax/_src/sharding_impls.py:826: error: Signature of "devices_indices_map" incompatible with supertype "Sharding"  [override]
+ jax/_src/sharding_impls.py:826: note:      Superclass:
+ jax/_src/sharding_impls.py:826: note:          def devices_indices_map(self, global_shape: tuple[int, ...]) -> Mapping[Device?, tuple[slice, ...] | None]
+ jax/_src/sharding_impls.py:826: note:      Subclass:
+ jax/_src/sharding_impls.py:826: note:          _lru_cache_wrapper[[GSPMDSharding, tuple[int, ...]], Mapping[Device?, tuple[slice, ...]]]
+ jax/_src/sharding_impls.py:827: error: Too few arguments  [call-arg]
+ jax/_src/sharding_impls.py:827: error: Argument 1 has incompatible type "tuple[int, ...]"; expected "GSPMDSharding"  [arg-type]
+ jax/_src/interpreters/mlir.py:575: error: Too few arguments  [call-arg]
+ jax/_src/interpreters/mlir.py:575: error: Argument 1 has incompatible type "Any | tuple[int | DArray | Tracer | Var | DBIdx | InDBIdx | OutDBIdx, ...]"; expected "XLACompatibleSharding"  [arg-type]
+ jax/_src/interpreters/pxla.py:1088: error: Too few arguments  [call-arg]
+ jax/experimental/multihost_utils.py:250: error: Too few arguments  [call-arg]
+ jax/experimental/multihost_utils.py:357: error: Too few arguments  [call-arg]
+ jax/experimental/jax2tf/shape_poly.py:1218: error: Too few arguments  [call-arg]
+ jax/experimental/jax2tf/shape_poly.py:1218: error: Argument 1 has incompatible type "int | Any"; expected "CachingShapeEvaluator"  [arg-type]
+ jax/experimental/jax2tf/shape_poly.py:1240: error: Too few arguments  [call-arg]
+ jax/experimental/jax2tf/shape_poly.py:1240: error: Argument 1 has incompatible type "int | Any"; expected "CachingShapeEvaluator"  [arg-type]
+ jax/experimental/jax2tf/shape_poly.py:1292: error: Too few arguments  [call-arg]
+ jax/experimental/jax2tf/shape_poly.py:1292: error: Argument 1 has incompatible type "Any | int"; expected "CachingShapeEvaluator"  [arg-type]
+ jax/experimental/jax2tf/shape_poly.py:1466: error: Too few arguments  [call-arg]
+ jax/experimental/array_serialization/serialization.py:53: error: Too few arguments  [call-arg]
+ jax/experimental/array_serialization/serialization.py:53: error: Argument 1 has incompatible type "tuple[int, ...]"; expected "XLACompatibleSharding"  [arg-type]
+ jax/experimental/array_serialization/serialization.py:258: error: Too few arguments  [call-arg]
+ jax/experimental/array_serialization/serialization.py:258: error: Argument 1 has incompatible type "tuple[Any, ...]"; expected "XLACompatibleSharding"  [arg-type]
+ jax/experimental/jax2tf/jax_export.py:943: error: Too few arguments  [call-arg]

materialize (https://github.com/MaterializeInc/materialize)
+ misc/python/materialize/mzbuild.py:634: error: Too few arguments  [call-arg]
+ misc/python/materialize/mzbuild.py:802: error: Too few arguments  [call-arg]
+ misc/python/materialize/cli/mzimage.py:39: error: Too few arguments  [call-arg]
+ misc/python/materialize/cli/mzimage.py:47: error: Too few arguments  [call-arg]

paroxython (https://github.com/laowantong/paroxython)
+ paroxython/assess_costs.py:85: error: Incompatible types in assignment (expression has type "_lru_cache_wrapper[[int, int], float]", variable has type "Callable[[LearningCostAssessor, int, int], float]")  [assignment]
+ paroxython/assess_costs.py:85: note: "_lru_cache_wrapper[[int, int], float].__call__" has type "Callable[[Arg(int, 'start'), Arg(int, 'stop')], float]"
+ paroxython/assess_costs.py:89: error: "Callable[[LearningCostAssessor, LearningCostAssessor, TaxonName], float]" has no attribute "cache_clear"  [attr-defined]
+ paroxython/assess_costs.py:128: error: Too few arguments  [call-arg]
+ paroxython/assess_costs.py:128: error: Argument 1 has incompatible type "TaxonName"; expected "LearningCostAssessor"  [arg-type]
+ paroxython/recommend_programs.py:296: error: Too few arguments  [call-arg]
+ paroxython/recommend_programs.py:296: error: Argument 1 has incompatible type "TaxonName"; expected "LearningCostAssessor"  [arg-type]
+ paroxython/map_taxonomy.py:243: error: Too few arguments  [call-arg]
+ paroxython/map_taxonomy.py:243: error: Argument 1 has incompatible type "LabelName"; expected "Taxonomy"  [arg-type]

jinja (https://github.com/pallets/jinja)
+ src/jinja2/environment.py:1186: error: Unused "type: ignore" comment  [unused-ignore]
+ src/jinja2/environment.py:1201: error: Unused "type: ignore" comment  [unused-ignore]

Daverball · 2023-08-05T17:27:25Z

Considering how hard you have to work with ParamSpec and Concatenate just to get to something that's close to being correct for some cases, but still not all of them, I'm inclined to believe that these just aren't the right tools for it, and it would be better suited to an Intersection and would be yet another use case where Intersection would really shine and keep things relatively simple by comparison.

So in this case rather than trying to return a different type from lru_cache you just return an Intersection between the callable that's been passed in and a Protocol which contains the additional properties that were added to the function. i.e. something like this:

_F = TypeVar("_F", bound=Callable[..., Any])

class _CacheMethods(Protocol):
    def cache_info(self) -> _CacheInfo: ...
    def cache_clear(self) -> None: ...
    if sys.version_info >= (3, 9):
        def cache_parameters(self) -> _CacheParameters: ...

if sys.version_info >= (3, 8):
    @overload
    def lru_cache(maxsize: int | None = 128, typed: bool = False) -> Callable[[_F], Intersection[_F, _CacheMethods]]: ...
    @overload
    def lru_cache(maxsize: _F, typed: bool = False) -> Intersection[_F, _CacheMethods]: ...

else:
    def lru_cache(maxsize: int | None = 128, typed: bool = False) -> Callable[[_F], Intersection[_F, _CacheMethods]]: ...

But maybe even this wouldn't work correctly, so not sure.

jakkdl · 2023-08-06T10:26:46Z

Considering how hard you have to work with ParamSpec and Concatenate just to get to something that's close to being correct for some cases, but still not all of them, I'm inclined to believe that these just aren't the right tools for it, and it would be better suited to an Intersection and would be yet another use case where Intersection would really shine and keep things relatively simple by comparison.

Yeah after digging thoroughly into this I have more or less given up for the moment. It might be possible to improve the stubs a little bit merely using Concatenate, but in general the tools seems insufficient for the job. Intersection sounds kinda perfect, but with the PEP only being WIP I'm assuming it's not coming with 3.12

JelleZijlstra · 2023-08-06T14:31:19Z

with the PEP only being WIP I'm assuming it's not coming with 3.12

Just FYI, assuming that PEP goes through, for typeshed usage the CPython release cycle doesn't really matter. The hypothetical timeline would go something like this:

The PEP is created and gets a number
typing_extensions.Intersection is added
Type checkers add support for intersections
The PEP is accepted
Typeshed starts using typing_extensions.Intersection
CPython implements the & operator for types (in 3.13 if this happens before ~May 2024, in 3.14 if after)

srittau · 2023-08-07T11:01:34Z

@jakkdl Thanks for your work on this so far. Should we close the PR for the time being?

jakkdl · 2023-08-07T19:01:20Z

@jakkdl Thanks for your work on this so far. Should we close the PR for the time being?

yeah, might as well :)

tamird · 2024-02-23T13:18:07Z

For the uninitiated: which PEP are you discussing above?

jakkdl · 2024-02-23T13:59:33Z

For the uninitiated: which PEP are you discussing above?

python/typing#213 (comment) iirc

Daverball · 2024-02-23T14:02:05Z

For the uninitiated: which PEP are you discussing above?

python/typing#213 (comment) iirc

Yes, exactly, the linked repository contains further information.

OrderedIntersection seems like it will get a PEP draft first however, courtesy of @mikeshardmind, read more about it here with the original pitch: CarliJoy/intersection_examples#38

mikeshardmind · 2024-02-24T03:11:30Z

👀 Let me add this issue to the list of motivating cases, thanks for the mention!

asvetlov and others added 7 commits July 31, 2023 17:50

Add typing for decorated function arguments in functools.cache() any …

89b145a

…functools.lru_cache()

Add missing types

0a965e2

Fix instancemethods

f612ee8

fix checher

a942bea

fix

cc45bd5

[pre-commit.ci] auto fixes from pre-commit.com hooks

e591a5b

implement suggested change from WMOkiishi

89e9326

This comment has been minimized.

Sign in to view

Reverting "implement suggested change from WMOkiishi" to see changes in

c179308

primer. This reverts commit 89e9326.

This comment was marked as resolved.

Sign in to view

jakkdl closed this Aug 7, 2023

sobolevn mentioned this pull request Feb 23, 2024

Possible option: use ParamSpec for lru_cache #6986

Closed

mikeshardmind mentioned this pull request Feb 24, 2024

Motivation section CarliJoy/intersection_examples#11

Open

jakkdl deleted the lru_cache-types-2 branch February 25, 2024 10:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

`functools`: Add typing for decorated function arguments in cache API [take 2] #10523

`functools`: Add typing for decorated function arguments in cache API [take 2] #10523

jakkdl commented Jul 31, 2023

This comment has been minimized.

github-actions bot commented Jul 31, 2023

This comment was marked as resolved.

Daverball commented Aug 5, 2023 •

edited

Loading

jakkdl commented Aug 6, 2023

JelleZijlstra commented Aug 6, 2023

srittau commented Aug 7, 2023

jakkdl commented Aug 7, 2023

tamird commented Feb 23, 2024

jakkdl commented Feb 23, 2024

Daverball commented Feb 23, 2024

mikeshardmind commented Feb 24, 2024

functools: Add typing for decorated function arguments in cache API [take 2] #10523

functools: Add typing for decorated function arguments in cache API [take 2] #10523

Conversation

jakkdl commented Jul 31, 2023

This comment has been minimized.

github-actions bot commented Jul 31, 2023

This comment was marked as resolved.

Daverball commented Aug 5, 2023 • edited Loading

jakkdl commented Aug 6, 2023

JelleZijlstra commented Aug 6, 2023

srittau commented Aug 7, 2023

jakkdl commented Aug 7, 2023

tamird commented Feb 23, 2024

jakkdl commented Feb 23, 2024

Daverball commented Feb 23, 2024

mikeshardmind commented Feb 24, 2024

`functools`: Add typing for decorated function arguments in cache API [take 2] #10523

`functools`: Add typing for decorated function arguments in cache API [take 2] #10523

Daverball commented Aug 5, 2023 •

edited

Loading