Skip to content

[WIP] typing improvements #6449

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from
Closed

[WIP] typing improvements #6449

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
8666b27
typing for AssertionRewritingHook, get_config, get_plugin_manager, _p…
blueyed Jan 13, 2020
604b84f
typing: get_parametrized_fixture_keys, Session._fixturemanager
blueyed Jan 15, 2020
b96077b
_PluggyPlugin
blueyed Jan 17, 2020
b95394b
typing: pytestconfig, request.config
blueyed Jan 17, 2020
0a8328a
# noqa: F821
blueyed Jan 17, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion src/_pytest/assertion/rewrite.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
format_explanation as _format_explanation,
)
from _pytest.compat import fspath
from _pytest.config import Config
from _pytest.pathlib import fnmatch_ex
from _pytest.pathlib import Path
from _pytest.pathlib import PurePath
Expand All @@ -39,7 +40,7 @@
class AssertionRewritingHook(importlib.abc.MetaPathFinder):
"""PEP302/PEP451 import hook which rewrites asserts."""

def __init__(self, config):
def __init__(self, config: Config) -> None:
self.config = config
try:
self.fnpats = config.getini("python_files")
Expand Down
18 changes: 14 additions & 4 deletions src/_pytest/config/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,8 @@
if TYPE_CHECKING:
from typing import Type

_PluggyPlugin = object

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bluetech since this might be used elsewhere - where should be put this?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As long is we keep it internal for now (I mean, not intended to be exposed in the "public" typing in the future), I guess putting it where PytestPluginManager is defined makes most sense?

A short comment here would be helpful, something like

A type to represent plugin objects. Plugins can be any namespace, so we can't narrow it down much, but we use an alias to make the intent clear. Ideally this type would be provided by pluggy itself.

BTW, I wouldn't put it under TYPE_CHECKING, this way we can use it without the annoying quotes.

Copy link
Contributor Author

@blueyed blueyed Jan 27, 2020 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Amended in a39b4e2 (#6580).


hookimpl = HookimplMarker("pytest")
hookspec = HookspecMarker("pytest")
Expand Down Expand Up @@ -172,7 +174,10 @@ def directory_arg(path, optname):
builtin_plugins.add("pytester")


def get_config(args=None, plugins=None):
def get_config(
args: Optional[Union[List, Tuple]] = None,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

List and Tuple without a [] is equivalent to List[Any] and Tuple[Any]. Are there narrower types that can be used?

If yes, it's OK to leave them out for now, leaving them out indicates a TODO.

If no, I prefer to put explicit List[Any], this it's clear it can really be anything and is not just omitted/TODO.

plugins: Optional[Sequence[Union[str, object]]] = None,
) -> "Config":
# subsequent calls to main will create a fresh instance
pluginmanager = PytestPluginManager()
config = Config(
Expand All @@ -191,7 +196,7 @@ def get_config(args=None, plugins=None):
return config


def get_plugin_manager():
def get_plugin_manager() -> PluginManager:
"""
Obtain a new instance of the
:py:class:`_pytest.config.PytestPluginManager`, with default plugins
Expand All @@ -203,7 +208,10 @@ def get_plugin_manager():
return get_config().pluginmanager


def _prepareconfig(args=None, plugins=None):
def _prepareconfig(
args: Optional[Union[List, Tuple]] = None,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this can also be a py.path.local. Even though it's not a real type yet, if you recall we decided to add it anyway so once typing is added to py.path it will start working.

plugins: Optional[Sequence[Union[str, "_PluggyPlugin"]]] = None,
):
if args is None:
args = sys.argv[1:]
elif isinstance(args, py.path.local):
Expand Down Expand Up @@ -732,7 +740,9 @@ class InvocationParams:
plugins = attr.ib()
dir = attr.ib(type=Path)

def __init__(self, pluginmanager, *, invocation_params=None):
def __init__(
self, pluginmanager: "PytestPluginManager", *, invocation_params=None
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe invocation_params: Optional[Config.InvocationParams]?

(OK if you left it out intentionally).

) -> None:
from .argparsing import Parser, FILE_OR_DIR

if invocation_params is None:
Expand Down
63 changes: 41 additions & 22 deletions src/_pytest/fixtures.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,10 @@
from collections import deque
from collections import OrderedDict
from typing import Dict
from typing import Generator
from typing import List
from typing import Tuple
from typing import Union

import attr
import py
Expand All @@ -29,6 +31,7 @@
from _pytest.compat import NOTSET
from _pytest.compat import safe_getattr
from _pytest.compat import TYPE_CHECKING
from _pytest.config import Config
from _pytest.deprecated import FIXTURE_POSITIONAL_ARGUMENTS
from _pytest.deprecated import FUNCARGNAMES
from _pytest.outcomes import fail
Expand All @@ -38,6 +41,9 @@
from typing import Type

from _pytest import nodes
from _pytest.main import Session
from _pytest.python import Class # noqa: F401
from _pytest.python import Function


@attr.s(frozen=True)
Expand All @@ -46,7 +52,7 @@ class PseudoFixtureDef:
scope = attr.ib()


def pytest_sessionstart(session):
def pytest_sessionstart(session: "Session") -> None:
import _pytest.python
import _pytest.nodes

Expand Down Expand Up @@ -179,30 +185,43 @@ def getfixturemarker(obj):
return None


def get_parametrized_fixture_keys(item, scopenum):
def get_parametrized_fixture_keys(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Depending on how much effort you're willing to expand on this function, it looks like this can be something like this:

@overload
def get_parametrized_fixture_keys(item: "nodes.Item", scopenum: Literal[0]) -> Iterator[Tuple[str, int]]: ...

@overload
def get_parametrized_fixture_keys(item: "nodes.Item", scopenum: Literal[1, 2]) -> Iterator[Tuple[str, int, py.path.local]]: ...

@overload
def get_parametrized_fixture_keys(item: "nodes.Item", scopenum: Literal[4]) -> Iterator[Tuple[str, int, py.path.local, "Class"]]: ...

Whether it needs scopenum: int fallback depends on the callers.

item: "nodes.Item", # only used for Function (with callspec), kept for B/C.
scopenum: int,
) -> Generator[
Union[
Tuple[str, int],
Tuple[str, int, py.path.local],
Tuple[str, int, py.path.local, "Class"],
],
None,
None,
]:
""" return list of keys for all parametrized arguments which match
the specified scope. """
assert scopenum < scopenum_function # function
assert scopenum < scopenum_function, (scopenum, scopenum_function)

try:
cs = item.callspec
cs = item.callspec # type: ignore[attr-defined] # noqa: F821
except AttributeError:
pass
else:
# cs.indices.items() is random order of argnames. Need to
# sort this so that different calls to
# get_parametrized_fixture_keys will be deterministic.
for argname, param_index in sorted(cs.indices.items()):
if cs._arg2scopenum[argname] != scopenum:
continue
return
if TYPE_CHECKING:
assert isinstance(item, Function)

# cs.indices.items() is random order of argnames. Need to
# sort this so that different calls to
# get_parametrized_fixture_keys will be deterministic.
for argname, param_index in sorted(cs.indices.items()):
if cs._arg2scopenum[argname] == scopenum:
if scopenum == 0: # session
key = (argname, param_index)
yield (argname, param_index)
elif scopenum == 1: # package
key = (argname, param_index, item.fspath.dirpath())
yield (argname, param_index, item.fspath.dirpath())
elif scopenum == 2: # module
key = (argname, param_index, item.fspath)
elif scopenum == 3: # class
key = (argname, param_index, item.fspath, item.cls)
yield key
yield (argname, param_index, item.fspath)
else: # class
assert scopenum == 3, scopenum
yield (argname, param_index, item.fspath, item.cls)


# algorithm for sorting on a per-parametrized resource setup basis
Expand Down Expand Up @@ -346,7 +365,7 @@ class FixtureRequest:
the fixture is parametrized indirectly.
"""

def __init__(self, pyfuncitem):
def __init__(self, pyfuncitem: "Function") -> None:
self._pyfuncitem = pyfuncitem
#: fixture for which this request is being performed
self.fixturename = None
Expand All @@ -355,7 +374,7 @@ def __init__(self, pyfuncitem):
self._fixture_defs = {} # type: Dict[str, FixtureDef]
fixtureinfo = pyfuncitem._fixtureinfo
self._arg2fixturedefs = fixtureinfo.name2fixturedefs.copy()
self._arg2index = {}
self._arg2index = {} # type: Dict[str, int]
self._fixturemanager = pyfuncitem.session._fixturemanager

@property
Expand Down Expand Up @@ -393,7 +412,7 @@ def _getnextfixturedef(self, argname):
return fixturedefs[index]

@property
def config(self):
def config(self) -> Config:
""" the pytest config object associated with this request. """
return self._pyfuncitem.config

Expand Down Expand Up @@ -1186,7 +1205,7 @@ def yield_fixture(


@fixture(scope="session")
def pytestconfig(request):
def pytestconfig(request: FixtureRequest) -> Config:
"""Session-scoped fixture that returns the :class:`_pytest.config.Config` object.

Example::
Expand Down
6 changes: 4 additions & 2 deletions src/_pytest/python.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,9 @@
from collections.abc import Sequence
from functools import partial
from textwrap import dedent
from typing import Dict
from typing import List
from typing import Optional
from typing import Tuple
from typing import Union

Expand Down Expand Up @@ -839,7 +841,7 @@ def __init__(self, metafunc):
self._globalparam = NOTSET
self._arg2scopenum = {} # used for sorting parametrized resources
self.marks = []
self.indices = {}
self.indices = {} # type: Dict[str, int]

def copy(self):
cs = CallSpec2(self.metafunc)
Expand Down Expand Up @@ -1368,7 +1370,7 @@ def __init__(
parent,
args=None,
config=None,
callspec=None,
callspec: Optional[CallSpec2] = None,
callobj=NOTSET,
keywords=None,
session=None,
Expand Down