How to test exceptions and tracebacks

[paddymul]

I am developing a Jupyter Widget that uses traitlets. My stack traces are occasionally huge (over 300 lines). I want to present the users of my widget with much more compact stacktraces only related to pieces that they control. I want to test that I'm properly filtering and limiting the stacktraces. I'm running into a couple problems trying to test this behaviour. Where should I look for inspiration?

I want to test that certain functions never end up in a traceback.

At this point I'm just asserting that the length of the traceback is less than what I'd get without the re-raising stuff i'm trying to test.

My tests look like this

import sys, traceback    
def tb_depth(tb, depth=1):
    " returns the depth of a traceback"
    if tb.tb_next is None:
        return depth
    else:
        return tb_depth(tb.tb_next, depth+1)

def test_exc_depth():

    def level_2():
        1/0
    def level_1():
        level_2()
    try:
        level_1()
    except Exception:
        l1_exc = sys.exception()
    assert tb_depth(l1_exc.__traceback__) == 3

1. It's hard to google for "test python exception traceback". I haven't found docs or libraries outside of traceback, which is part of the solution, but I'm not completely sure how to use it to achieve the result I want
2. The stack traces and frames that I see when I inspect in the middle of a pytest test has a lot of pluggy and pytest stuff

```python-traceback (, 115) (, 493) (, 1792) (, 169) (, 77) (, 115) (, 493) ```

3. Jupyter notebook does its own things with exceptions and tracebacks for interactive code. I never run this code as a standalone python module

I made a small example notebook showing the stacktraces I don't want and the cleaned up ones
https://gist.github.com/paddymul/acffe2a902db7ad2cee1a7c223be7f88

[nicoddemus]

Hey,

I want to test that I'm properly filtering and limiting the stacktraces.

How are you doing that? I ask because pytest will filter out any traceback frame which contains a local variable __tracebackhide__ = True set.

At this point I'm just asserting that the length of the traceback is less than what I'd get without the re-raising stuff i'm trying to test.

I would probably parse each file name out of each stack frame, trying to somehow find out the module to which the frame belongs to, and then decide it based on a block/allow list (for example "pytest" in Path(fn).parts). This is from the top off my head, I'm sure it will eventually give some false positives, but I find it less brittle than testing the traceback length, as that can easily change (both your own code and external code).