-
-
Notifications
You must be signed in to change notification settings - Fork 31.6k
assertRaises increases reference counter #68078
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Sometimes unittest's assertRaises increases reference counter of callable. This can break tests in tricky cases. Not seen in 2.X version. Demo file attached. |
This is presumably a result of the exception object being saved on the context manager object and there being a circular reference chain that doesn't immediately get GCed. This is just how python works. I don't know if it would be sensible/useful to use the new lightweight traceback stuff in assertRaises. If so it would probably have to be optional for backward compatibility reasons. |
It seems, as a minimum it must be noticed in docs. |
I don't think we make any guarantees for or against references, but - we attempt to free references already (see how we call clear_frames), so this is a bug, pure and simple. |
As Robert noted, this is a straight up error, where the clear_frames() call and deleting the exception traceback from the saved object aren't enough to drop all references to The trick is that the cleanup code isn't accounting for __cause__ and __context__: it's only clearing and dropping the traceback for the topmost exception in the chain. In Vjacheslav's example, there are *two* tracebacks to worry about: both the top level one (which is getting cleaned up) and the inner one from exc.__context__ which isn't being touched. We could likely use a "traceback.clear_all_frames()" helper that walks an exception tree clearing *all* the traceback frames, both on the original exception, and on all the __cause__ and __context__ attributes. (We can't just clear cause and context, since those can be accessed and tested when using the context manager form and the |
I've been looking into this further, and a reproducer that's independent of assertRaises() is to just bind the function to a local variable name in an outer function: def outer():
callable_obj = f
try:
callable_obj()
except Exception as exc:
return exc That should make it easier to test a basic recursive "clear_all_frames" operation like: def clear_all_frames(exc):
cause = exc.__cause__
if cause is not None:
clear_all_frames(cause)
context = exc.__context__
if context is not None:
clear_all_frames(context)
traceback.clear_frames(exc.__traceback__) |
Victor's PR takes a more pragmatic approach to address this problem: rather than adding the ability to clear the frames for an arbitrary exception tree, it just uses a try/finally in a couple of key unittest function definitions to clear the offending circular references by setting them to None. |
It seems like Python 2.7 is not affected. |
Thank you Vjacheslav Fyodorov for your bug report! The bug should now be fixed in 3.5, 3.6 and master (future 3.7) branches. Python 2.7 is not affected by the bug. |
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
bugs.python.org fields:
The text was updated successfully, but these errors were encountered: