unable to determine the correct type for pytest.mark.parametrize

[renzhentaxi]

Describe the bug
pyright is unable to determine the correct type for pytest's mark.parametrize decorator

To Reproduce
setup:
os: linux
python 3.8.5,
pytest 6.1.2
pyright 1.1.86

pyrightconfig.json:

{
	"typeCheckingMode": "strict",
	"reportUnusedVariable": "information"
}

Expected behavior
pyright shouldn't throw the following errors (if its pyright's fault)

3:2 - error: Type of "parametrize" is unknown (reportUnknownMemberType)
3:2 - error: Untyped function decorator obscures type of function; ignoring decorator (reportUntypedFunctionDecorator)

Screenshots or Code
If applicable, add screenshots or the text of the code (surrounded by triple back ticks) to help explain your problem.

import pytest

@pytest.mark.parametrize("param", ["dog", "cat"])
def test_sample(param: str):
    assert False

VS Code extension or command-line
running pyright directly via command line, latest version (1.1.86)

Additional context
not sure if it helps but mypy fixed the decorator issue here: python/mypy#9232
pytest-dev/pytest#8053

[erictraut]

This is a problem with the pytest package. It claims to be typed (i.e. it contains a "py.typed" file), but it includes some variables that are not annotated. The parametrize variable is one such example. As such, pyright is treating it as an "unknown" type.

For more details about how pyright treats py.typed packages, refer to this documentation. We're trying to standardize this behavior across type checkers so package maintainers will see consistent behavior and understand what is expected of them.

In this case, pyright is doing the right thing. Please file bugs or PR in the pytest repo.

[erictraut]

Incidentally, we're seeing more packages that claim to be typed but contain incomplete type information. Pytest is not alone here. We recently added a mode to the CLI version of pyright that is able to analyze a package and report its "type completeness". I just ran "pyright --verifytypes pytest" on the latest pytest package. Pyright reports that there are 49 public symbols, and 28 of them are missing some or all type information. That translates to a type completeness score of 42.9%.

[bluetech]

(pytest dev coming from the pytest issue)

pytest.mark.parametrize is actually typed, although it's under if TYPE_CHECKING: ... inside a class scope: https://github.com/pytest-dev/pytest/blob/afd53ede6f3c950df3bfc9e81afdd45e09ef6d2b/src/_pytest/mark/structures.py#L476. Maybe pyright doesn't handle this?

[erictraut]

It looks like the annotations were added recently. The latest published version of pytest (6.1.2) does not annotate these variables. Here's what I'm seeing:

    if TYPE_CHECKING:
        # TODO(py36): Change to builtin annotation syntax.
        skip = _SkipMarkDecorator(Mark("skip", (), {}))
        skipif = _SkipifMarkDecorator(Mark("skipif", (), {}))
        xfail = _XfailMarkDecorator(Mark("xfail", (), {}))
        parametrize = _ParametrizeMarkDecorator(Mark("parametrize", (), {}))
        usefixtures = _UsefixturesMarkDecorator(Mark("usefixtures", (), {}))
        filterwarnings = _FilterwarningsMarkDecorator(Mark("filterwarnings", (), {}))

So it looks like this issue will be fixed once the next version of pytest is published. That's great news!

[bluetech]

Huh, I forgot about that. So pyright doesn't infer a type from the assignment? In any case if it works with the annotation syntax that's great, I'll report that in the pytest issue.

[erictraut]

If a package claims to be typed, then pyright will not attempt to apply inference. The rules for inference are not defined in any Python standards, and each type checker applies slightly different rules for inference. If type checkers were to apply inference for py.typed packages, you'd see different results. For that reason, it's important for py.typed packages to include complete type annotations.

[renzhentaxi]

Yep. Can confirm that it works once the annotation is added