Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add option to selectively disable --disallow-untyped-calls #15845

Merged
merged 4 commits into from Aug 13, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
34 changes: 34 additions & 0 deletions docs/source/command_line.rst
Expand Up @@ -350,6 +350,40 @@ definitions or calls.
This flag reports an error whenever a function with type annotations
calls a function defined without annotations.

.. option:: --untyped-call-exception

This flag allows to selectively disable :option:`--disallow-untyped-calls`
for functions and methods defined in specific packages, modules, or classes.
Note that each exception entry acts as a prefix. For example:

.. code-block:: python

# mypy --disallow-untyped-calls --untyped-call-exception=foo --untyped-call-exception=bar.A
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe also use a real third-party package in the example, such as numpy?

from foo import test_foo
from bar import A, B
from baz import test_baz

test_foo(42) # OK, function comes from module `foo`
test_baz(42) # E: Call to untyped function "test_baz" in typed context

a: A
b: B
a.meth() # OK, method was defined in class `bar.A`
b.meth() # E: Call to untyped function "meth" in typed context

# file foo.py
def test_foo(x): pass

# file bar.py
class A:
def meth(self): pass
class B:
def meth(self): pass

# file baz.py
def test_baz(x): pass


.. option:: --disallow-untyped-defs

This flag reports an error whenever it encounters a function definition
Expand Down
32 changes: 31 additions & 1 deletion docs/source/config_file.rst
Expand Up @@ -490,7 +490,37 @@ section of the command line docs.
:default: False

Disallows calling functions without type annotations from functions with type
annotations.
annotations. Note that when used in per-module options, it enables/disables
this check **inside** the module(s) specified, not for functions that come
from that module(s), for example config like this:

.. code-block:: ini

[mypy]
disallow_untyped_calls = True

[mypy-some.library.*]
disallow_untyped_calls = False

will disable this check inside ``some.library``, not for your code that
imports ``some.library``. If you want to selectively disable this check for
all your code that imports ``some.library`` you should instead use
:confval:`untyped_call_exception`, for example:

.. code-block:: ini

[mypy]
disallow_untyped_calls = True
untyped_call_exception = some.library

.. confval:: untyped_call_exception
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems useful, but I'm not sure about the name of the option. Maybe we can come up with a better name? I don't any suggestions right now, but I'll think about this later.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also struggled some time with this. I would propose to not spend much time on this. People who want this will probably search for "mypy untyped call" or similar, so including these words in the option name should be good enough. So if you have a better idea, then I will be happy to update, otherwise let's just move on.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fwiw I'd go with untyped_call_allowlist. Clearly conveys the type as well ;-)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is what I considered at first, but then how wold we call corresponding command line option? --untyped-call-allowlist? All other options like --always-true, --always-false, --enable-error-code, --disable-error-code etc, are singular and should be repeated. So I would say we should try to follow this pattern. Finally, from reading comments in the issue, it looks like most people will need just 1 or maybe 2 exceptions, not more.


:type: comma-separated list of strings

Selectively excludes functions and methods defined in specific packages,
modules, and classes from action of :confval:`disallow_untyped_calls`.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mention that this also applies to submodules of packages (i.e. everything inside that prefix).

Note, this option does not support per-file configuration, the exception
list is defined globally for all your code.

.. confval:: disallow_untyped_defs

Expand Down
22 changes: 15 additions & 7 deletions mypy/checkexpr.py
Expand Up @@ -529,13 +529,6 @@ def visit_call_expr_inner(self, e: CallExpr, allow_none_return: bool = False) ->
callee_type = get_proper_type(
self.accept(e.callee, type_context, always_allow_any=True, is_callee=True)
)
if (
self.chk.options.disallow_untyped_calls
and self.chk.in_checked_function()
and isinstance(callee_type, CallableType)
and callee_type.implicit
):
self.msg.untyped_function_call(callee_type, e)

# Figure out the full name of the callee for plugin lookup.
object_type = None
Expand All @@ -561,6 +554,21 @@ def visit_call_expr_inner(self, e: CallExpr, allow_none_return: bool = False) ->
):
member = e.callee.name
object_type = self.chk.lookup_type(e.callee.expr)

if (
self.chk.options.disallow_untyped_calls
and self.chk.in_checked_function()
and isinstance(callee_type, CallableType)
and callee_type.implicit
):
if fullname is None and member is not None:
assert object_type is not None
fullname = self.method_fullname(object_type, member)
if not fullname or not any(
fullname.startswith(p) for p in self.chk.options.untyped_call_exception
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sure the exception re doesn't apply to retry, i.e. the startswith operation should probably be at module name component level.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

):
self.msg.untyped_function_call(callee_type, e)

ret_type = self.check_call_expr_with_callee_type(
callee_type, e, fullname, object_type, member
)
Expand Down
18 changes: 18 additions & 0 deletions mypy/config_parser.py
Expand Up @@ -81,6 +81,20 @@ def validate_codes(codes: list[str]) -> list[str]:
return codes


def validate_package_allow_list(allow_list: list[str]) -> list[str]:
for p in allow_list:
msg = f"Invalid allow list entry: {p}"
if "*" in p:
raise argparse.ArgumentTypeError(
f"{msg} (entries are already prefixes so must not contain *)"
)
if "\\" in p or "/" in p:
raise argparse.ArgumentTypeError(
f"{msg} (entries must be packages like foo.bar not directories or files)"
)
return allow_list


def expand_path(path: str) -> str:
"""Expand the user home directory and any environment variables contained within
the provided path.
Expand Down Expand Up @@ -164,6 +178,9 @@ def split_commas(value: str) -> list[str]:
"plugins": lambda s: [p.strip() for p in split_commas(s)],
"always_true": lambda s: [p.strip() for p in split_commas(s)],
"always_false": lambda s: [p.strip() for p in split_commas(s)],
"untyped_call_exception": lambda s: validate_package_allow_list(
[p.strip() for p in split_commas(s)]
),
"enable_incomplete_feature": lambda s: [p.strip() for p in split_commas(s)],
"disable_error_code": lambda s: validate_codes([p.strip() for p in split_commas(s)]),
"enable_error_code": lambda s: validate_codes([p.strip() for p in split_commas(s)]),
Expand All @@ -187,6 +204,7 @@ def split_commas(value: str) -> list[str]:
"plugins": try_split,
"always_true": try_split,
"always_false": try_split,
"untyped_call_exception": lambda s: validate_package_allow_list(try_split(s)),
"enable_incomplete_feature": try_split,
"disable_error_code": lambda s: validate_codes(try_split(s)),
"enable_error_code": lambda s: validate_codes(try_split(s)),
Expand Down
17 changes: 16 additions & 1 deletion mypy/main.py
Expand Up @@ -11,7 +11,12 @@
from typing import IO, Any, Final, NoReturn, Sequence, TextIO

from mypy import build, defaults, state, util
from mypy.config_parser import get_config_module_names, parse_config_file, parse_version
from mypy.config_parser import (
get_config_module_names,
parse_config_file,
parse_version,
validate_package_allow_list,
)
from mypy.errorcodes import error_codes
from mypy.errors import CompileError
from mypy.find_sources import InvalidSourceList, create_source_list
Expand Down Expand Up @@ -675,6 +680,14 @@ def add_invertible_flag(
" from functions with type annotations",
group=untyped_group,
)
untyped_group.add_argument(
"--untyped-call-exception",
metavar="MODULE",
action="append",
default=[],
help="Disable --disallow-untyped-calls for functions/methods coming"
" from specific package, module, or class",
)
add_invertible_flag(
"--disallow-untyped-defs",
default=False,
Expand Down Expand Up @@ -1307,6 +1320,8 @@ def set_strict_flags() -> None:
% ", ".join(sorted(overlap))
)

validate_package_allow_list(options.untyped_call_exception)

# Process `--enable-error-code` and `--disable-error-code` flags
disabled_codes = set(options.disable_error_code)
enabled_codes = set(options.enable_error_code)
Expand Down
4 changes: 4 additions & 0 deletions mypy/options.py
Expand Up @@ -136,6 +136,10 @@ def __init__(self) -> None:
# Disallow calling untyped functions from typed ones
self.disallow_untyped_calls = False

# Always allow untyped calls for function coming from modules/packages
# in this list (each item effectively acts as a prefix match)
self.untyped_call_exception: list[str] = []

# Disallow defining untyped (or incompletely typed) functions
self.disallow_untyped_defs = False

Expand Down
51 changes: 51 additions & 0 deletions test-data/unit/check-flags.test
Expand Up @@ -2077,6 +2077,57 @@ y = 1
f(reveal_type(y)) # E: Call to untyped function "f" in typed context \
# N: Revealed type is "builtins.int"

[case testDisallowUntypedCallsAllowListFlags]
# flags: --disallow-untyped-calls --untyped-call-exception=foo --untyped-call-exception=bar.A
from foo import test_foo
from bar import A, B
from baz import test_baz

test_foo(42) # OK
test_baz(42) # E: Call to untyped function "test_baz" in typed context

a: A
b: B
a.meth() # OK
b.meth() # E: Call to untyped function "meth" in typed context
[file foo.py]
def test_foo(x): pass
[file bar.py]
class A:
def meth(self): pass
class B:
def meth(self): pass
[file baz.py]
def test_baz(x): pass

[case testDisallowUntypedCallsAllowListConfig]
# flags: --config-file tmp/mypy.ini
from foo import test_foo
from bar import A, B
from baz import test_baz

test_foo(42) # OK
test_baz(42) # E: Call to untyped function "test_baz" in typed context

a: A
b: B
a.meth() # OK
b.meth() # E: Call to untyped function "meth" in typed context
[file foo.py]
def test_foo(x): pass
[file bar.py]
class A:
def meth(self): pass
class B:
def meth(self): pass
[file baz.py]
def test_baz(x): pass

[file mypy.ini]
\[mypy]
disallow_untyped_calls = True
untyped_call_exception = foo, bar.A

[case testPerModuleErrorCodes]
# flags: --config-file tmp/mypy.ini
import tests.foo
Expand Down