New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pytest 3.0.2 memory leak with pytest.raises #1965

Closed
MSeifert04 opened this Issue Sep 26, 2016 · 7 comments

Comments

Projects
None yet
3 participants
@MSeifert04
Contributor

MSeifert04 commented Sep 26, 2016

  • Description

I tried to identify a memory leak in one of my functions and while debugging it I realized that besides the memory leak in my function there is also a memory leak in pytest.raises.

  • conda list

colorama 0.3.7
pip 8.1.2 py35_0
py 1.4.31
pytest 3.0.2
python 3.5.2 0
setuptools 27.2.0 py35_1
vs2015_runtime 14.0.25123 0
wheel 0.29.0 py35_0

Windows 10 (64 bit) but the memory leak is also visible in Travis CI ( Linux 64 bit) and AppVeyor (I think 64 bit) independant of python version. The memory leak is not present in pytest 2.9.2 and 2.6.4 (tested with Travis CI)

  • Minimal example

It's not really minimal and there are probably better ways to test it:

from collections import Counter
from gc import get_objects
import pytest

def difference_tracked_objects(func, specific_object=None):
    # rather trivial attempt at finding memory leaks, this attempt does not always work! 
    # That's why I created a custom class and used the "specific_object".
    before = Counter()
    after = Counter()
    before.update(map(type, get_objects()))
    func()
    after.update(map(type, get_objects()))
    if specific_object is None:
        return after - before
    else:
        leftover = after[specific_object] - before[specific_object]
        if leftover:
            return Counter({specific_object: leftover})
        else:
            return Counter()

class Test(object):
    def __init__(self, value):
        self.value = value

def testfunc():
    func = lambda x: x.value + ''
    v = Test(10)
    with pytest.raises(TypeError):
        func(v)

difference_tracked_objects(testfunc, Test)
# Counter() (on pytest < 3.0.2)
# Counter({<class '__main__.Test'>: 1}) (on pytest == 3.0.2) <-- looks like memory leak
@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Sep 27, 2016

Thanks for the report!

@MSeifert04

This comment has been minimized.

Contributor

MSeifert04 commented Oct 17, 2016

Is there any way I can assist in fixing this issue?

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Oct 17, 2016

You might want to take a look at the code and perhaps submitting a PR? The relevant code for that is in python.py.

@MSeifert04

This comment has been minimized.

Contributor

MSeifert04 commented Oct 17, 2016

@nicoddemus Thank you for the pointer. I submitted a PR but I think I might need some help creating a regression test. At least locally the changes seem to fix the issue.

However it's really weird that Python can't garbage the cyclic references - normally that's not a problem.

@RonnyPfannschmidt

This comment has been minimized.

Member

RonnyPfannschmidt commented Oct 18, 2016

@MSeifert04 does happen after triggering a gc.collect?

i recall that at least on cpython full gc is only triggered on occasion because quite often recounting is more than sufficient

@MSeifert04

This comment has been minimized.

Contributor

MSeifert04 commented Oct 18, 2016

i recall that at least on cpython full gc is only triggered on occasion because quite often recounting is more than sufficient

You're right gc.collect fixes the problem. 😅

@nicoddemus

This comment has been minimized.

Member

nicoddemus commented Oct 18, 2016

Strange, I could sworn I tried gc.collect()... oh well, glad it worked in the end. 😁

mozillazg pushed a commit to mozillazg/pypy that referenced this issue Nov 29, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment