Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MRG+1] TST remove temp files and folders #2570

Merged
merged 2 commits into from Feb 20, 2017
Merged

[MRG+1] TST remove temp files and folders #2570

merged 2 commits into from Feb 20, 2017

Conversation

@kmike
Copy link
Member

@kmike kmike commented Feb 16, 2017

This PR makes sure temp files are deleted in tests. It also fixes a weird pytest+OS X issue. Previously tests failed for me with exception like this:

______________________________________________ ERROR at setup of WebClientTestCase.test_Encoding ______________________________________________

cls = <class 'py._path.local.LocalPath'>, prefix = 'test_Encoding'
rootdir = local('/private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gn/T/pytest-of-kmike/pytest-0'), keep = 0, lock_timeout = None

    def make_numbered_dir(cls, prefix='session-', rootdir=None, keep=3,
                          lock_timeout = 172800):   # two days
        """ return unique directory with a number greater than the current
                maximum one.  The number is assumed to start directly after prefix.
                if keep is true directories with a number less than (maxnum-keep)
                will be removed.
            """
        if rootdir is None:
            rootdir = cls.get_temproot()

        def parse_num(path):
            """ parse the number out of a path (if it matches the prefix) """
            bn = path.basename
            if bn.startswith(prefix):
                try:
                    return int(bn[len(prefix):])
                except ValueError:
                    pass

        # compute the maximum number currently in use with the
        # prefix
        lastmax = None
        while True:
            maxnum = -1
            for path in rootdir.listdir():
                num = parse_num(path)
                if num is not None:
                    maxnum = max(maxnum, num)

            # make the new directory
            try:
>               udir = rootdir.mkdir(prefix + str(maxnum+1))

/Users/kmike/svn/scrapy/.tox/py36/lib/python3.6/site-packages/py/_path/local.py:825:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/Users/kmike/svn/scrapy/.tox/py36/lib/python3.6/site-packages/py/_path/local.py:459: in mkdir
    py.error.checked_call(os.mkdir, fspath(p))
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <py._error.ErrorMaker object at 0x110844518>, func = <built-in function mkdir>
args = ('/private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gn/T/pytest-of-kmike/pytest-0/test_Encoding0',), kwargs = {}
__tracebackhide__ = False, cls = <class 'py.error.EEXIST'>, value = FileExistsError(17, 'File exists'), tb = <traceback object at 0x117b3f248>
errno = 17

    def checked_call(self, func, *args, **kwargs):
        """ call a function and raise an errno-exception if applicable. """
        __tracebackhide__ = True
        try:
            return func(*args, **kwargs)
        except self.Error:
            raise
        except (OSError, EnvironmentError):
            cls, value, tb = sys.exc_info()
            if not hasattr(value, 'errno'):
                raise
            __tracebackhide__ = False
            errno = value.errno
            try:
                if not isinstance(value, WindowsError):
                    raise NameError
            except NameError:
                # we are not on Windows, or we got a proper OSError
                cls = self._geterrnoclass(errno)
            else:
                try:
                    cls = self._geterrnoclass(_winerrnomap[errno])
                except KeyError:
                    raise value
>           raise cls("%s%r" % (func.__name__, args))
E           py.error.EEXIST: [File exists]: mkdir('/private/var/folders/_5/cbsg50991szfp1r9nwxpx8580000gn/T/pytest-of-kmike/pytest-0/test_Encoding0',)
kmike added 2 commits Feb 16, 2017
pytest intercepts temp file creation to provide readable file/folder
names; path is built from method name; in case of conflicts
pytests uses increasing numbers, but it seems it doesn’t account
for case-insensitive (but case preserving) OS X filesystem. There
are methods named test_encoding, pytest thinks test_Encoding is
different and fails to create a test folder
@kmike kmike changed the title Tests cleanup resources TST remove temp files and folders Feb 16, 2017
@kmike
Copy link
Member Author

@kmike kmike commented Feb 16, 2017

It seems this is the pytest bug we're observing: pytest-dev/pytest#708

@kmike kmike mentioned this pull request Feb 16, 2017
1 of 1 task complete
@redapple redapple changed the title TST remove temp files and folders [MRG+1] TST remove temp files and folders Feb 20, 2017
@kmike kmike merged commit 7ad5156 into master Feb 20, 2017
1 check passed
1 check passed
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@dangra dangra deleted the tests-cleanup-resources branch Feb 20, 2017
@dangra dangra added this to the v1.4 milestone Feb 20, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants