API for excluding methods from unittest stack traces

[rbtcollins]

BPO	1705520
Nosy	@rhettinger, @amauryfa, @rbtcollins, @voidspace, @davidmalcolm, @Julian, @serhiy-storchaka

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/voidspace'
closed_at = None
created_at = <Date 2007-04-23.03:56:38.000>
labels = ['type-feature', 'library', '3.11']
title = 'API for excluding methods from unittest stack traces'
updated_at = <Date 2021-09-05.17:45:39.872>
user = 'https://github.com/rbtcollins'

bugs.python.org fields:

activity = <Date 2021-09-05.17:45:39.872>
actor = 'serhiy.storchaka'
assignee = 'michael.foord'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2007-04-23.03:56:38.000>
creator = 'rbcollins'
dependencies = []
files = []
hgrepos = []
issue_num = 1705520
keywords = []
message_count = 9.0
messages = ['55078', '61493', '116666', '116667', '122753', '123620', '123634', '123635', '401099']
nosy_count = 8.0
nosy_names = ['rhettinger', 'purcell', 'amaury.forgeotdarc', 'rbcollins', 'michael.foord', 'dmalcolm', 'Julian', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'needs patch'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue1705520'
versions = ['Python 3.11']

[rbtcollins]

pyunit looks for __unittest = 1 in the globals of functions in an assertion stacktrace. This is used to trim the trace so that unittest implementation methods do not show up.

It would be great if it looked in the locals of the frame as well, as this would allow subclasses of TestCase that add new assertFOO methods to have them filtered in the same manner.

e.g. (bad example :))
def assertComplexState(self, inputs):
__unittest = 1
self.assertEqual('42', inputs[0], 'the input %s is not the right answer' % inputs)

[purcell]

This is a good idea, but slightly messy.

A better solution would be for unittest to provide a decorator that
could be used to mark assertion functions. Internally, that decorator
may well work the way you describe.

Given a corresponding patch, someone with more time than myself may well
be willing to commit it.

[BreamoreBoy]

is this still relevant?

[voidspace]

It is relevant and would be *possible* to implement. I'm not 100% convinced it is a good *enough* idea to make it worth adding though. I'd like to leave the issue open for the moment in case other people want to comment.

[voidspace]

__unittest needs to die (with appropriate deprecation).

I agree that a nicer API for marking functions and methods as needing to be excluded from unittest stacktraces would be useful. A decorator is a good way to expose that API to developers. Internally unittest could use the new API and __unittest can go away.

Introspecting stack frames is a bad way to implement this, as it doesn't play well with other implementations (like IronPython which doesn't have introspectable Python stack frames by default).

[voidspace]

So from the stackframe you can only get to the code object not to the function object and although the code object is also reachable from a decorator it isn't mutable so we can't "mark it" in any way. We could in theory 're-build' the function and create a new code object (code is assignable) by copying the original but adding a non-existent name to one of the code attributes. This is pretty horrible though.

So a *real* local variable (as suggested by Robert) would work, but the functionality wouldn't work on IronPython and would cause tools like pyflakes to complain about an unused name.

Another alternative is to use the docstring. This is available via code_object.co_consts[0], so we could use a marker in the docstring to indicate that this frame should be omitted from tracebacks. Unless we can make the marker either invisible (magic whitespace anyone?) or non-invasive this is also a not so pleasant idea. (And wouldn't work with -OO.)

Alternative suggestions welcomed.

[amauryfa]

Maybe a global registry... implemented with a WeakKeyDictionary?

[voidspace]

Global registry of code objects, hmmm... Could work. I'll prototype it and test it with IronPython / Jython / pypy.

[serhiy-storchaka]

What to do with comprehensions and classes? Corresponding code objects are not easily accessible and they do not have corresponding function. It would be difficult to use the locals of the frame with comprehensions.

Maybe use per-module registries of qualnames?

class MyAssertions:
    def assertComplexState(self, inputs):
        self.assertEqual('42', inputs[0], 'the input %s is not the right answer' % inputs)

__unittests = {'MyAssertions.assertComplexState'}

The frame is skipped if f_globals['__unittests'] contains co_qualname or any parents of co_qualname.

We can even add a decorator:

def assertion(func):
    mod = sys.modules[func.__module__]
    mod.__dict__.setdefault('__unittests', set())
    mod.__setdefault.add(func.__qualname__)
    return func