Cleanup remote tracebacks

[j-bennet]

Adds a utility function shorten_traceback, and wraps sys.excepthook to apply it to unhandled exceptions.

The behavior of the function is controlled by several settings:

• admin.traceback.shorten.when
• admin.traceback.shorten.what

If both are non-empty, and any of the stack frames match a pattern listed in admin.traceback.shorten.when, stack trace is shortened. Frames that match a pattern listed in admin.traceback.shorten.what are omitted. The method always preserves the first and last stack frame though.

For IPython, instead of sys.excepthook, a custom exception handler is provided.

The second part - adding worker info to the remote exception - is implemented in dask/distributed#7847.

Issue: dask/distributed#4880.

• Tests added / passed
• Passes pre-commit run --all-files

As discussed with @crusaderky.

[j-bennet]

@crusaderky Finally got the test passing. Lesson learned: never expect ipython to behave the same on different envs. This is now ready for review.

One more thing I was thinking about adding to this PR: exposing a function to install the excepthook, instead of modifying it without the user participation. It might be an overkill. What do you think?

[crusaderky]

@j-bennet see my review commit
Merging as soon as you confirm you're happy with my changes

[crusaderky]

exposing a function to install the excepthook, instead of modifying it without the user participation. It might be an overkill. What do you think?

I don't think I follow. What function are you referring to?

[j-bennet]

@crusaderky

@j-bennet see my review commit Merging as soon as you confirm you're happy with my changes

when and what sounds fine.

I don't think I follow. What function are you referring to?

In the PR at the current state, as soon as the user imports something from dask, sys.excepthook is wrapped, even if the wrapper does nothing when admin.traceback.shorten is false.

Alternatively, wrapping excepthook could be something that's triggered by the user, by calling a method, let's say, dask.instal_hooks.

[crusaderky]

In the PR at the current state, as soon as the user imports something from dask, sys.excepthook is wrapped, even if the wrapper does nothing when admin.traceback.shorten is false.

Alternatively, wrapping excepthook could be something that's triggered by the user, by calling a method, let's say, dask.instal_hooks.

I see your reasoning, but I feel that if we added a completely different API it would make things worse for the UX rather than better.
This (and quite a few other situations) could potentially resolved by being able to install callbacks in dask.config.set. Definitely out of scope though.

[crusaderky]

Thank you!