New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Incorrect number of collected items reported when specific class methods are provided #2464

Closed
tonyo opened this Issue Jun 2, 2017 · 4 comments

Comments

Projects
None yet
3 participants
@tonyo

tonyo commented Jun 2, 2017

  • Include a detailed description of the bug or suggestion

It seems that the number of collected tests is displayed incorrectly when the "class method" notation (ModuleX::ClassY::MethodZ) is used from the command line. In a simple example (see below) when we want to collect/run only one specific test method from a class, one expects to get the message that only 1 item was collected (collected 1 item). Instead, pytest outputs collected N items, where N equals the total number of test methods collected from ClassY plus one.

  • pip list of the virtual environment you are using
appdirs (1.4.3)
packaging (16.8)
pip (9.0.1)
py (1.4.33)
pyparsing (2.2.0)
pytest (3.1.2.dev13+gf826b23, /home/user/reps/pytest_fix/pytest)
setuptools (35.0.2)
six (1.10.0)
wheel (0.30.0a0)
  • pytest and operating system versions
pytest version 3.1.2.dev13+gf826b23

Ubuntu 16.04.2, Linux 4.4.0-78-generic x86_64
  • Minimal example if possible
# collect.py

class TestClass(object):
    def test_main(self):
        assert 1
    def test_bla(self):
        assert True

$ py.test collect.py::TestClass::test_main --collect-only

Output:

=================== test session starts ======================
platform linux2 -- Python 2.7.12, pytest-3.1.2.dev13+gf826b23, py-1.4.33, pluggy-0.4.0
rootdir: /home/user/reps/pytest_fix, inifile:
collected 3 items 
<Module 'collect.py'>
  <Class 'TestClass'>
    <Instance '()'>
      <Function 'test_main'>

Expected output:

...
collected 1 item 
<Module 'collect.py'>
...
@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Jun 2, 2017

Just to be clear here: the problem is that it should display collected 2 items since there are 2 items in TestClass, althought it will end up running one of them.

@tonyo

This comment has been minimized.

tonyo commented Jun 2, 2017

@nicoddemus Is it documented somewhere? I expected --collect-only to return the number of tests that pytest would run if started without --collect-only.

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Jun 2, 2017

I expected --collect-only to return the number of tests that pytest would run if started without --collect-only.

That's perfectly reasonable to assume, it it is just not how things work currently.

To discover test items from a module, pytest has to import that module. At that point it creates the test items it finds in that namespace, and only then it picks out the test id provided in the command line to execute. It does so because it has to create a test item in order to know its node id (to account for parametrization, hooks, etc, anything which in the end affects the node id), so it might as well create all test items for a module upon discovery.

nicoddemus added a commit to nicoddemus/pytest that referenced this issue Jun 3, 2017

nicoddemus added a commit to nicoddemus/pytest that referenced this issue Jun 3, 2017

nicoddemus added a commit to nicoddemus/pytest that referenced this issue Jun 3, 2017

@tonyo

This comment has been minimized.

tonyo commented Jun 5, 2017

@nicoddemus Thank you for the explanation, it makes more sense now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment