New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing error: atexit cleanup functions -> psutils, (isolated example in body) #93

Closed
bionikspoon opened this Issue Dec 8, 2015 · 3 comments

Comments

Projects
None yet
2 participants
@bionikspoon
Contributor

bionikspoon commented Dec 8, 2015

Testing errors with redislite, using pytest and tox

When testing: I'm getting a series of errors that start at redislite's atexit cleanup methods and all lead back to a line in psutil:

Error in atexit._run_exitfuncs:
Traceback (most recent call last):
...
...
...
  File "/home/travis/build/bionikspoon/isolate_redislite_tox_psutil/.tox/pypy/site-packages/psutil/_pslinux.py", line 137, in get_procfs_path
    return sys.modules['psutil'].PROCFS_PATH
KeyError: 'psutil'
debug: OperationError:
debug:  operror-type: KeyError
debug:  operror-value: 'psutil'
ERROR: InvocationError: '/home/travis/build/bionikspoon/isolate_redislite_tox_psutil/.tox/pypy/bin/python setup.py test'

Except for with pypy, the tests are still passing; but they raise the error in the reports--full tracebacks linked in the gists below. With pypy the tests are failing completely.

I created an isolated example to recreate the error--in the form of a github repo (linked below). It mimics the setup I'm using in a minimalist fashion.


Full Tracebacks

Debug Info

Isolated Example


Finally, I feel obligated to offer some solution--even though I don't fully understand that cause of the problem: offer a cleanup utility function that we can call in the test "teardown" phase.

@bionikspoon

This comment has been minimized.

Show comment
Hide comment
@bionikspoon

bionikspoon Dec 8, 2015

Contributor

Alright, I found the cause and a variety of solutions. The cause is best explained in this article: https://briandamaged.org/blog/setuptools-test-command-hates-atexit/ . In summary, the issue is caused specifically by using setup.py to run your test command. setuptools deletes loaded modules as part of a cleanup routine.

Solutions:

There's one detailed in the article.

I posted a very simple solution in a comment (awaiting moderation). You can run the atexit cleanup routine before setuptools starts deleting modules by calling atexit._run_exitfuncs(). Gist. Result: All passing tests; no error message. This is what I'm going to do for now on my project.

In some cases: you can manually run the RedisMixin._cleanup method as part of the "tear down" phase. I demonstrate this, in a branch of that isolated example I posted: test_example.py. Result: All passing tests; no error message. Though, this didn't seem to work on another project.


All these solutions feel very "hackish" and don't help in terms of not forwarding the problem to your end user. I hope there's a good solution to this. If not, maybe some documentation on the issue to help the next guy might be useful.

Contributor

bionikspoon commented Dec 8, 2015

Alright, I found the cause and a variety of solutions. The cause is best explained in this article: https://briandamaged.org/blog/setuptools-test-command-hates-atexit/ . In summary, the issue is caused specifically by using setup.py to run your test command. setuptools deletes loaded modules as part of a cleanup routine.

Solutions:

There's one detailed in the article.

I posted a very simple solution in a comment (awaiting moderation). You can run the atexit cleanup routine before setuptools starts deleting modules by calling atexit._run_exitfuncs(). Gist. Result: All passing tests; no error message. This is what I'm going to do for now on my project.

In some cases: you can manually run the RedisMixin._cleanup method as part of the "tear down" phase. I demonstrate this, in a branch of that isolated example I posted: test_example.py. Result: All passing tests; no error message. Though, this didn't seem to work on another project.


All these solutions feel very "hackish" and don't help in terms of not forwarding the problem to your end user. I hope there's a good solution to this. If not, maybe some documentation on the issue to help the next guy might be useful.

@dwighthubbard

This comment has been minimized.

Show comment
Hide comment
@dwighthubbard

dwighthubbard Dec 14, 2015

Contributor

I think we need to do something for this and I appreciate all the info in this issue. I think for the near term we should document this and for the longer term we need a fix since I've already noticed the failures in pypy.

Contributor

dwighthubbard commented Dec 14, 2015

I think we need to do something for this and I appreciate all the info in this issue. I think for the near term we should document this and for the longer term we need a fix since I've already noticed the failures in pypy.

dwighthubbard added a commit that referenced this issue Mar 4, 2016

Merge pull request #94 from bionikspoon/patch-1
Fixes #93  Make sure `psutil` is available
@bionikspoon

This comment has been minimized.

Show comment
Hide comment
@bionikspoon

bionikspoon Mar 4, 2016

Contributor

\o/

Contributor

bionikspoon commented Mar 4, 2016

\o/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment