Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix test collection from packages mixed with directories. #3768 and #3789 #3802

Merged
merged 3 commits into from Aug 16, 2018

Conversation

@jonozzz
Copy link

commented Aug 10, 2018

This should fix the issue but I still can't find a better fix. See the comment in the code.

What happens is: the files and directories are marked as duplicates when the Session starts collection even though they're not turned into nodes right away (the package nodes will do that later). Ideally is that when Session finds a Package it should stop recursing and move on.

@coveralls

This comment has been minimized.

Copy link

commented Aug 10, 2018

Coverage Status

Coverage increased (+0.07%) to 92.542% when pulling abae60c on jonozzz:fix-3768 into 4d8903f on pytest-dev:master.

Copy link
Member

left a comment

I'm a little puzzled by the need to handle duplicated paths in Package.collect.

But we really need tests that replicate #3768 and #3789 to prove the patch works and avoid regressions in the future. Please do let me know if you have trouble with the tests. 👍

@@ -216,18 +216,6 @@ def pytest_pycollect_makemodule(path, parent):
return Module(path, parent)


def pytest_ignore_collect(path, config):

This comment has been minimized.

Copy link
@nicoddemus

nicoddemus Aug 10, 2018

Member

Yeah this does not really seem necessary, main.pytest_ignore_collect already seems to do the exactly same thing anyway.

This comment has been minimized.

Copy link
@jonozzz

jonozzz Aug 10, 2018

Author

Initially I copy/pasted the Package class from Session because they both serve very similar functions, and then realized not everything was needed.

@@ -594,6 +580,15 @@ def isinitpath(self, path):
return path in self.session._initialpaths

def collect(self):
# XXX: HACK!

This comment has been minimized.

Copy link
@nicoddemus

nicoddemus Aug 10, 2018

Member

Strange, I expect main.pytest_ignore_collect to handle this properly already:

pytest/src/_pytest/main.py

Lines 261 to 284 in 4d8903f

def pytest_ignore_collect(path, config):
ignore_paths = config._getconftest_pathlist("collect_ignore", path=path.dirpath())
ignore_paths = ignore_paths or []
excludeopt = config.getoption("ignore")
if excludeopt:
ignore_paths.extend([py.path.local(x) for x in excludeopt])
if py.path.local(path) in ignore_paths:
return True
allow_in_venv = config.getoption("collect_in_virtualenv")
if _in_venv(path) and not allow_in_venv:
return True
# Skip duplicate paths.
keepduplicates = config.getoption("keepduplicates")
duplicate_paths = config.pluginmanager._duplicatepaths
if not keepduplicates:
if path in duplicate_paths:
return True
else:
duplicate_paths.add(path)
return False

If pytest_ignore_collect sees the same path twice, it should ignore the collection when see the path the second time, which would not trigger a new Package item to be collected.

This comment has been minimized.

Copy link
@jonozzz

jonozzz Aug 10, 2018

Author

The problem is it sees that some modules were already "collected" (at Session level) via the recurse() method. Then when the package goes and tries to collect its children and turn them into nodes it can't because they look like duplicates. Wrong! My first reaction was to try to stop marking them as duplicates at Session level, but everything is so tangled up that I can't find a cleaner way.

This comment has been minimized.

Copy link
@blueyed

blueyed Oct 25, 2018

Contributor

JFI: I've moved it in #4241.



def test_package_with_modules(testdir):
"""

This comment has been minimized.

Copy link
@nicoddemus

nicoddemus Aug 11, 2018

Member

Just FYI, you could have created the actual files in testing/example_scripts and copied the whole directory here like the code above this function does.

The idea is that it is easier to execute the test using pytest as normal during testing/debugging. 😉

This is fine as is, just thought I would mention this for the future. 👍

@nicoddemus

This comment has been minimized.

Copy link
Member

commented Aug 11, 2018

LGTM! Took the liberty of committing the missing changelog entries. 👍

@nicoddemus nicoddemus merged commit 64faa41 into pytest-dev:master Aug 16, 2018
2 checks passed
2 checks passed
continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
blueyed added a commit to blueyed/pytest that referenced this pull request Oct 25, 2018
This removes the hack added in
pytest-dev#3802.
blueyed added a commit to blueyed/pytest that referenced this pull request Oct 25, 2018
This removes the hack added in pytest-dev#3802.

Adjusts test:

- it appears to not have been changed to 7 intentionally.
- removes XXX comment, likely not relevant anymore since 6dac774.
blueyed added a commit to blueyed/pytest that referenced this pull request Nov 1, 2018
This removes the hack added in pytest-dev#3802.

Adjusts test:

- it appears to not have been changed to 7 intentionally.
- removes XXX comment, likely not relevant anymore since 6dac774.

Backport of e041823 (pytest-dev#4241)
from features.
blueyed added a commit to blueyed/pytest that referenced this pull request Nov 1, 2018
This removes the hack added in pytest-dev#3802.

Adjusts test:

- it appears to not have been changed to 7 intentionally.
- removes XXX comment, likely not relevant anymore since 6dac774.

Backport of e041823 (pytest-dev#4241)
from features.
blueyed added a commit to blueyed/pytest that referenced this pull request Nov 1, 2018
This removes the hack added in pytest-dev#3802.

Adjusts test:

- it appears to not have been changed to 7 intentionally.
- removes XXX comment, likely not relevant anymore since 6dac774.

Backport of e041823 (pytest-dev#4241)
from features.
kchmck added a commit to kchmck/pytest that referenced this pull request Nov 1, 2018
This removes the hack added in pytest-dev#3802.

Adjusts test:

- it appears to not have been changed to 7 intentionally.
- removes XXX comment, likely not relevant anymore since 6dac774.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.