Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

High memory usage if tests called by test case name #1521

Open
fontealpina opened this issue Apr 18, 2016 · 9 comments
Open

High memory usage if tests called by test case name #1521

fontealpina opened this issue Apr 18, 2016 · 9 comments
Labels
topic: collection related to the collection phase type: performance performance or memory problem/improvement

Comments

@fontealpina
Copy link

fontealpina commented Apr 18, 2016

I'm experiencing an high memory usage if the tests to run are called by test case name instead of test file name.
To explain my issue I will use the following trivial test, called test_try.py:

class TestSuite:
    def test_case_1(self):
        assert True

    def test_case_2(self):
        assert True

    def test_case_3(self):
        assert True

If I want to run all the tests, I can type:

$ pytest -v test_try.py
======================== test session starts ========================
collected 3 items

test_try.py::TestSuite::test_case_1 PASSED
test_try.py::TestSuite::test_case_2 PASSED
test_try.py::TestSuite::test_case_3 PASSED
====================== 3 passed in 0.01 seconds ====================

But in my workflow may be useful call the tests by test case name, so:

$ pytest -v test_try.py::TestSuite::test_case_1 test_try.py::TestSuite::test_case_2 test_try.py::TestSuite::test_case_3
======================== test session starts ========================
collected 12 items

NuSMV/test/bug_026/test_try.py::TestSuite::test_case_1 PASSED
NuSMV/test/bug_026/test_try.py::TestSuite::test_case_2 PASSED
NuSMV/test/bug_026/test_try.py::TestSuite::test_case_3 PASSED
====================== 3 passed in 0.01 seconds ====================

In the first case there are 3 collected items, while in the second case the collected items are 12.
I guess that for every test case there are 4 test collected(3 for all the test case in the file and one for the real test case to run).

If this approach is applied on a larger scale (for instance with many test files that contains many test), I observed a huge increase of memory usage.

I was wandering if there is an easy way to collect only the single test case in order to lower the number of test collected and so the memory usage?

Thanks

@RonnyPfannschmidt RonnyPfannschmidt added type: bug problem that needs to be addressed topic: collection related to the collection phase labels Apr 18, 2016
@RonnyPfannschmidt
Copy link
Member

can you outline used pytest-plugins and py.test versions as well?

at first glance i fear, that we run collection multiple times on the same file, i'll try to verify this week

@fontealpina
Copy link
Author

ok I tried both with:

platform linux2 -- Python 2.6.6, pytest-2.9.2.dev1, py-1.4.26, pluggy-0.3.1 -- /usr/bin/python
platform linux2 -- Python 2.7.6, pytest-2.9.2.dev1, py-1.4.26, pluggy-0.3.1 -- /usr/bin/python

but also with an old version:

platform linux2 -- Python 2.6.6 -- py-1.4.26 -- pytest-2.7.0.dev1 -- /usr/bin/python

@RonnyPfannschmidt
Copy link
Member

thanks for verifying that the issue is with python 2.6 (which is EOL)

@fontealpina
Copy link
Author

Wait a moment.
I'm sorry I've updated the message because I was checking the issue in python2.7 in a wrong way.
So I have updated the message.
I'm obtaining the same issue both with python2.6 and python2.7.
Sorry

@fontealpina
Copy link
Author

Any news about this issue?
Thanks

@RonnyPfannschmidt
Copy link
Member

i didn't find the time to look at it again so far

@fontealpina
Copy link
Author

Sorry, have you checked this?
To me would be enough even a temporary fix to avoid it, maybe can you suggest a way to do it.
Thanks

@RonnyPfannschmidt
Copy link
Member

i still didnt manage to look into detail

my first idea idea for fixing it is to go in, and split node-id based collection, into a plugin that inhibits collection outside of the requested items

it would be a class which implements a make-item hook wrapper and discards undesirable results
and it would be installed into the plugin-manager, if such tests where given as arguments

@nicoddemus
Copy link
Member

@RonnyPfannschmidt that sounds like a good idea. Perhaps @fontealpina can implement that hook into his conftest file so we can be sure the solution will work? I would suggest implementing pytest_collect_file by looking at the path component of the test id instead if pytest_pycollect_makeitem is too complicated.

@Zac-HD Zac-HD added type: performance performance or memory problem/improvement and removed type: bug problem that needs to be addressed labels Jan 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
topic: collection related to the collection phase type: performance performance or memory problem/improvement
Projects
None yet
Development

No branches or pull requests

4 participants